SCA, SBOM, Vulnerability Management, SAST, or DAST Tools: Which Is Best for Your Team?

There are a lot of options for software security testing tools. How do you know which ones are right for you? Some types of tools, such as SCA tools, are made to find vulnerabilities in existing code, while others, such as DAST tools, are more useful for finding vulnerabilities in your own code. Some tools only find potential vulnerabilities, while others find confirmed vulnerabilities.

Which should you choose, and more importantly, why?

‍

In this blog post, I'm going to cover a simple two-step process that will allow you to pick the best software security tool for your organization. 

Step One: Third party code or your own code?

When it comes to software security analysis, the first step is deciding whether to prioritize finding vulnerabilities in third-party code or your own code. You have two choices: 

1. Look for vulnerabilities in third-party code, where the main goal is to look for an update from the original author, or
2. Look for vulnerabilities in the code you write with the main goal of fixing the vulnerability 

‍

Finding Vulnerabilities in Third-party Code (Your Dependencies)

Third-party code, such as open source and third-party dependencies, is different. The main security concern is the third-party code has a known vulnerability. For example, if you depend upon OpenSSL for cryptography, you want to know if the particular version you are using has known flaws. 

SCA and SBOM tools are popular because they help you identify if the version of software you are using has any known vulnerabilities. These tools have two parts: a database of known vulnerable version numbers and a scanner to check what versions are present on your system.   For example, the database may contain an entry that python library lxml version 4.6.0 has known vulnerability PYSEC-2021-19. The scanner checks your python programs dependencies by reading requirements.txt and seeing if libxml version 4.6.0 is included.  

These tools are great at detecting if your software is essentially out of date. But they also have three limitations:

1. They don’t find unknown vulnerabilities. They only index known vulnerabilities.
2. They don’t know if the code is on the attack surface. They just know the vulnerable version string is present.
3. They miss vulnerabilities if they cannot detect or find what version numbers are used. As a result, SCA tools are most useful for languages and scenarios that have an index of all used dependencies, such as Python and Javascript.

‍

Finding Vulnerabilities in Your Code

When you prioritize finding vulnerabilities in your own code, you also are deciding to look for unknown vulnerabilities. It’s your own code, after all, so any vulnerabilities are original to you. The analysis tools you choose will work differently than third-party tools like SCA. They need to understand how your own code works in order to infer where it may be broken or vulnerable. 

Step Two: “Might be” or “Must be”?

The second step is deciding what confidence level you want in the vulnerabilities your software detects.

Would you rather:

1. Know several possible vulnerabilities in your code, none of which are confirmed. All of them could be real, or none of them could be real. I call this “might be” analysis.
2. Know one confirmed vulnerability. You’ve seen it exploited. I call this “must be” analysis.

Unfortunately, there is no free lunch that would give you the best of both worlds. It’s an iron law of computer science. You need to choose.

"Might Be" Vulnerabilities

If you decide you are a “might be” person, you’ll want a tool that will over-approximate the real risks you might have. SAST is a great fit for you. Commercial SAST tries to over-approximate what may happen and show you as many possible vulnerabilities as it can.

"Must Be" Vulnerabilities 

If you decide you are a “must be” person, you’ll want a tool that will confirm problems before reporting them. If this is you, dynamic analysis (DAST) is a better fit. DAST tools, and I don’t just mean web scanners but any dynamic tool, verifies problems happen in runtime. But they only see code that actually runs.

The obvious question: couldn’t you run both? Of course. But before you do, think about this: once you run both tools, what will you base your decision on? These tools don’t work together, where DAST confirms SAST. Anyone who says otherwise is just trying to market to you.

Let's Review Your Four Options: 

1. Third-party code, might-be analysis. Choose an SCA tool.
2. Third-party code, must-be analysis. Choose a vulnerability scanner.
3. Your code, might-be analysis. Choose a static analysis tool.
4. Your code, must-be analysis (my favorite). Choose a DAST tool like Mayhem.

‍

{{code-cta}}