The Buyer’s Guide to Mayhem and Comprehensive Application Security

What Makes Software Vulnerable?

Software bugs are born when detailed programs are built with mistakes. These bugs are viewed as defects until malicious activity exploits them as vulnerabilities. Though, applications may have software vulnerabilities for a variety of reasons including:

Mistakes: Developers live and die by the roadmap, but their mission to quickly deliver creative features may make mistakes that harm users.

Misconfiguration: Application stacks are complex. This makes misconfigurations of application components lead to vulnerabilities. An instance of these components is frameworks to databases to servers to platform.

Inherited Vulnerability: Free, open source and third-party code is more popular than ever with development demands. Free, however, is never free. An organization’s benefit of speed and scale could be paying a price in security. Organizations can inherit security risks from the unchecked application supply chain.

‍Neglect: Application verification and validation best practices call for positive testing and negative testing. The objective of negative testing is to ensure the software remains stable in unexpected use cases and is free of vulnerabilities.

The application attack surface is growing by 111 billion new lines of code every year, with newly reported zero-day exploits rising from one-per-week in 2015 to one-per day in 2021.

‍

How are Vulnerabilities Identified?

‍

Application vulnerabilities fall into one of three risk categories:

Known Known: Known Knowns are identifiable risks that are known to lead to compromise. These risks are identified through a Common Vulnerabilities and Exposure (CVE) ID.

Known Unknown: Known Unknowns are identifiable risks that could potentially lead to compromise, and these risks exhibit software flaw patterns that are likely to create exploitable vulnerabilities. These risks are identified through a Common Weakness Enumeration (CWE) ID.

Unknown Unknown: Unknown Unknowns are risks that cannot be identified. Unknown Unknowns present the greatest risk, because they enable adversaries to operate unnoticed for an extended period of time.

In the sections following, consider the detailed technologies that fall within these two categories and outline how these tests could minimizing software security risk in your organization.

Navigating AppSec Solutions

A myriad of security tools creates the foundation software security best practices.Organizations need different security tools to satisfy their unique needs. By and large, most software security solutions fall into one of two categories:

Secure the Source: Integrate application security testing tools through the SDLC to find vulnerabilities and weakness before deployment. An example of Secure the Source is Static Code Analysis (SAST), which works as prevention during the development stage.

Find and Fix: Identify security vulnerabilities and misconfigurations in production applications for development teams to fix later. An example of Find and Fix is Dynamic Analysis (DAST), which works as detection during quality assurance.

‍

Static Application Security Testing (SAST)

‍

Static application security testing (also known as static analysis or static code analysis)tools uncover bugs by analyzing source code. The defects they identify are known unknown risks. SAST vendors dissect each CWE to uncover specifications. Those specifications are then implemented to detect code flaws and weaknesses that could lead to vulnerabilities. This is a long, laborious process. Thus, SASTs, as with most application security testing (AST) tools, are better as they age. More experience and knowledge built into SASTs products can catch more defects.

SASTs operate in the world of “what-ifs”, taking in potentially relevant information to make assumptions on what “could-be”. The cautious nature of SASTs processes with good intent, but this approach leads to high false-positives and inaccuracies for many organizations.Many organizations gripe about SAST’s inaccuracy. All businesses lack time and resources, and therefore, it is unrealistic for them to chase every potential issue.

Despite its shortcomings, SASTs have their place in the SDL as recommended, preventative practice. Simply put, it’s good hygiene. Since static analysis require source code, SASTs are able to provide prescriptive remediation advice, down to the line of code. Static analysis can also be introduced earlier than most tools in the software development lifecycle, lowering cost and effort for remediation.

A vendor with a market leading SAST solution revealed that, on average, it takes an enterprise approximately 8 years for their code to be deemed “clean”.

‍

Dynamic Application Security Testing (DAST)

Dynamic application security testing is also known as dynamic analysis. Unlike SAST, DAST serves as an overarching category for a variety of techniques. To provide more specificity, we’ll focus on a DAST technique known as software composition analysis (SCA), a software security testing solution that nicely complements SAST.

Some, though not all, SCAs can uncover vulnerabilities without access to source code. The vulnerabilities they identify are known known risks. When vulnerabilities become known,SCAs have automated capabilities that pull information from the National VulnerabilitiesDatabase (NVD) to detect them in the software they’re scanning.

SCAs operate in the world of “what-is”, relying on accepted and publicly known information to uncover vulnerabilities. While they prevent adversaries from exploiting low-hanging fruits, they foster a reactive security approach -- meaning they mitigate risk after the window of exposure has opened. When vulnerabilities become known, they are publicly disclosed to organizations and adversaries at the same time. By nature, vulnerabilities are frictionless weapons. Vulnerabilities can be turned around and used in nefarious ways, at scale. Thus, time is of the essence when remediating these vulnerabilities.

SCAs offer prescriptive advice, listing the affected component and providing a link to its patch. This level of actionability is unique in DAST solutions. However, bear in mind, not all known vulnerabilities flagged by an SCA solution is exploitable. Additionally, each patch much be tested to ensure interoperability with the entirety of the application and/or the ecosystem they live in.

^‍

Mayhem for Security Testing

Mayhem is a new technique under the dynamic testing and grey-box testing category. The defects Mayhem identifies are unknown unknown risks. Mayhem uses advanced fuzz testing techniques to uncovers defects utilizing unknown or uncommon attack patterns. After each simulated attack, it monitors and leverages its target reactions, or behaviors, as feedback to autonomously generate new test cases that are increasingly likely to uncover more defects and new code edges.

Mayhem operates between the world of “what-if” and “what-is”. It uncovers unknown defects, enabling organizations to be preventative and proactive. Interestingly, continuous fuzz testing like Mayhem has been a proven and accepted software security practice for years. However, fuzzing has been exclusive to technology behemoths, such as Google, Microsoft, Apple, Nvidia, and more, who have the technical savvy and budget to implement and maintain such technologies.

Latest advancements in this field of study have dramatically improved usability, makingMayhem increasingly accessible to everyone. Although dynamic testing has required complete builds for testing, Mayhem has defied these inherent limitations with its capability to conduct regression and component testing during development, allowing organizations to find issues early, often, and continuously as a part of their DevOps pipeline.

Evan Johnson, Head of Product Security at Cloudflare, points out how easily Mayhem can start finding software vulnerabilities. “The learning curve of new tools can be steep, and Mayhem made it as frictionless as possible for our engineers to start finding bugs and gain more assurance in our software,” Johnson shares.

Security engineers of Google’s OSS-Fuzz team have disclosed that while it is possible to bootstrap and operate high-performance fuzzers, people often underestimate the complexity of upstanding such solutions. Yet, the benefit of fuzzing is undeniable.Teams at Google report that fuzzing uncovers 80% of their bugs, with the other 20%uncovered by other testing techniques, or naturally in production.

‍

True to the DAST technique, Mayhem is accurate and precise, uncovering defects with zero false positives. As mentioned prior, Mayhem is a grey-box solution, which means it conducts dynamic testing with visibility into source code. It can pinpoint the exact line of affected code and provides expert remediation advice, making fixes as simple as possible for developers.

All Together Now

Secure development practices call for the use of various testing techniques throughout the development lifecycle. SAST, SCA, and Mayhem strategically offer strength in each technique’s limitations, offering comprehensive application testing across the spectrum of software security risk.

Solutions for Unknown Risks

SAST for Known Unknown Risk

SAST directly analyzes the code to detect coding and design vulnerabilities and/or weaknesses. There is a focus on quality with a low number of false-positives. Test during the SDLC development phase with manual test case creation.

• Non-running state during testing.

• Low to Medium Accuracy.

• Low automation.

• No supply chain management.

• Unconfirmed exploitability of found bugs.

• Doesn’t check every line of code.

• Fast interaction of loop

‍

Mayhem for Unknown Unknown Risk

Mayhem analyses software for vulnerabilities by monitoring for anomalous behavior to expected inputs. There is a focus on security and stability with zero false-positives. Test during the SDLC development or QA phase with high automation.

• Running state during testing.

• High Accuracy.

• High automation.

• Supply chain management.

• Confirmed exploitability of found bugs.

• Doesn’t check every line of code.

• Fast interaction of loop

‍

Knowing What’s In It For You

There are many benefits to employing a multi-pronged software security approach:

• Lower costs due to early vulnerability prevention and/or detection.

• Greater software quality, security, and stability.

• Security testing for right- and left-of-ship.

• Secured and hardened attack surface for minimized risk.

• Detection of known, unknown, and zero-day vulnerabilities.

‍

Now, It’s Your Turn

Software is everywhere. It permeates nearly every aspect of our lives and has proven itself to be a productivity enhancer. As our dependency on software increase, we must ensure their reliability.

A multi-pronged application security strategy comprised of static analysis, software composition analysis, and continuous fuzz testing offer comprehensive security testing coverage across the spectrum of software risks. Well-known testing techniques like SAST andSCA are not enough as they can leave significant gaps behind.

Mayhem’s continuous fuzz testing is a proven and accepted method for uncovering unknown defects automatically. It offers strengths in SAST and SCA’s limitations. Utilizing a combination of these three security testing techniques not only ensure that organizations are enabled to deliver safe, secure, reliable software and service, but also implements predictability in today’s unpredictable, ever-changing threat landscape.

‍