Successful Coverage with Mayhem For API

Once authentication is configured, the next step is to ensure that Mayhem for API is able to successfully cover as much of your API as possible.

What is Successful Coverage?

Mayhem for API considers any 2xx response from the API server to indicate success, and a single successful response from a given endpoint is enough for us to consider it successfully covered.

If your API has some other criteria for success: we don't currently support it, but we are interested in this use case, so please reach out to us on Discord or by emailing mayhem4api@forallsecure.com, and let's see what we can figure out!

Why is Successful Coverage Important?

The absence of findings, for an endpoint which is not successfully covered, is meaningless.

Mayhem for API fuzzing is designed around this idea: that the most interesting test results come from requests that are almost correct. Some of the payloads we generate are purely random—that's important, too!—but once we find a successful payload for a given endpoint, we use that as a starting point for many future requests.

Consider a single POST endpoint which executes an INSERT statement against a relational database on success. If every single request issued by Mayhem for API is invalid, the INSERT statement has never been executed; if that INSERT statement has a SQL injection vulnerability, it won't be found.

If, on the other hand, we pass a valid payload (causing the INSERT statement to be executed) and get a 200 from this endpoint, we'll then use the successful payload as the basis for many future tests. If that INSERT statement has a SQL injection bug, we'll eventually find it.

That's why we emphasize "successful coverage": successfully covering an endpoint massively increases our confidence that the absence of findings is meaningful.

Reasons For Lack of Successful Coverage

Authentication, which is documented separately, is often the first barrier to successful coverage.

Mayhem for API (and fuzzing in general) usually does just fine, without intervention, generating values that satisfy simple data validations—things like "the 'page' parameter must be an integer".

What are other reasons Mayhem for API might be failing to successfully cover an endpoint?

Incorrect URLs

Sometimes the simplest solution is hard to see! If you're getting unexpected failures on endpoints that should be easy to exercise successfully—especially if the failures are 404s!—double-check that Mayhem for API is hitting the correct URLs.

Complex Structural Validation

ℹ️ If you start with thorough OpenAPI specs, you're less likely to run into this problem.

API Security. Performance. Validation. Fast.

Prime Your APIs for Performance ... In As Little As 5 Minutes.

Get Free Request A Demo

While we'll fairly quickly generate a random value that "must be an integer", it'll take quite a long time to generate a random value that, for example, "must be a JSON document with exactly one key, named 'page', whose value is an integer".

Stateful Validation

The trickiest validations are ones which are based on the current state of the service. For example, a value that "must be the unique id of an existing row in the 'pages' table" might take essentially forever to generate through pure randomness.

Mayhem for API has some tricks up its sleeves (such as trying values from the output of one request in the inputs of another) that help, but it's not perfect.

Whatever the reason, what follows are suggestions for diagnosing and fixing problems in successful coverage.

Diagnosing Problems in Successful Coverage

If you're still here, that means you have at least one endpoint that Mayhem for API can't yet successfully cover. So, how do you begin to figure out why that's happening?

It's worth noting that this may be extremely hard unless you either are (or working closely with someone who is) deeply familiar with the service being tested.

Step One: Focus

We strongly recommend that you dig into one endpoint (or a very small set of interrelated endpoints) at a time, when attempting to get successful coverage, and to keep the runs fairly short (60 seconds or less).

Filter your runs down via the --include-endpoint <endpoint> flag to the mapi run.

Step Two: Logs

When looking at logs, what you're looking for are patterns: what kinds of requests is Mayhem for API making, and why are they failing? What kinds of requests is Mayhem for API not making, and how can it be coerced into making them?

Start by looking at your own service logs, since hopefully you're familiar with those!

If your own logs aren't illuminating enough, you can also capture the full request/response payloads of an entire fuzzing run using the --har <file> flag to mapi run. The output is in JSON HAR format.

Step Three: Reach Out!

Don't hesitate to reach out to us on Discord or by emailing mayhem4api@forallsecure.com. Between your expertise with your service, and our expertise with Mayhem for API, we should be able to figure it out!