Zero Findings Is Not Zero Flaws

“SAST and DAST both came back clean on this release, so it is safe to ship.”

Clean against what?

That question is the whole article.

Zero findings is not zero flaws.

A findings count tells you what a tool examined and recognised.

It says nothing about what the tool never examined.

And it says nothing about what the tool could not recognise even while looking straight at it.

Picture the report for a DAST scan that reached your billing workflow and found nothing.

Now picture the report for a DAST scan that never got past the login page in front of your billing workflow.

They are the same report.

Zero. Green. Ship it.

Nobody would accept “the smoke detector in the garage is silent, so the kitchen is not on fire.”

Replace the smoke detector with a scanner licence and the kitchen is suddenly certified.

Each scanner category has a blind spot built into what it observes.

Not a maturity gap waiting for next year’s release.

A category boundary.

And the measures secure development uses to score itself, maturity models and tool adoption, describe whether a practice exists.

Not whether this release is free of a flaw.

Where those two meet, an empty findings list quietly merges two very different states.

Not tested.

Tested clean.

Your release gate cannot tell them apart unless you make it.

1. “But We Scan Every Release With Both SAST and DAST.”

Good.

Now name what each one cannot see.

SAST reads source, bytecode or a build artifact without running it.

It can in principle trace every line that exists, including paths that never run in production.

What it characteristically lacks is runtime context: reachability, sanitization and architectural controls.

That is where authorization flaws live.

Take an insecure direct object reference.

The code that fetches the record does exactly what it was written to do.

The query is well formed.

Nothing untrusted flows into a dangerous sink.

The missing element is a business decision: does this session’s user own this record?

That exists as a runtime fact and as a requirement.

It does not exist as a pattern in the source.

A SAST tool can flag the absence of any authorization check around a sensitive operation, if that is a rule it was built with.

It cannot judge whether an existing check checks the right thing.

“Right” lives in the requirements, not the syntax.

So the scariest authorization bug in your release is often the one with a check.

Just the wrong check.

It produces no finding, because nothing about it looks missing.

DAST sends crafted requests to the running application and reads the responses.

It can confirm a finding is exploitable against the deployed system, including misconfiguration that never appears in source.

It can also only test what its crawler found or what its configuration points at.

An endpoint no crawled page links to is never exercised.

Neither is a multi-step authenticated workflow the scanner was not configured to walk.

Neither is anything behind a login flow with a CSRF token or an MFA step the tool was not given the credentials and logic to complete.

That is a coverage problem, not an accuracy problem.

The scanner did not get it wrong.

It never went there.

And the report does not distinguish “never went there” from “went there, found nothing.”

OWASP’s Top 10 2021 data puts a number on how uneven this is, even for a well-studied category.

Server-Side Request Forgery carries a reported testing coverage of 67.72% across the applications contributing data.

Subtract that from 100 and roughly a third, 32.28%, had no confirmed test coverage for the category at all.

“We run DAST” and “we tested for SSRF” are different sentences.

2. “But the Scanner Is Configured With a Test Account.”

One account.

Configured with one role.

Walking the pages that role can reach.

That fixes the login page.

It does not fix the workflow three steps past it, unless someone scripted those steps.

And it does not generate the request that matters most for authorization.

Your authenticated account asking for another account’s resource.

A crawler exploring as a single user sees that user’s world.

It has no reason to request a record it was never shown.

Generic test suites are not systematically built to generate cross-account or cross-tenant attempts either.

IAST looks like the fix here, and it narrows the gap.

An agent inside the application’s runtime sees execution paths and data flows during real requests.

That is runtime context SAST does not have.

But IAST inherits DAST’s constraint in a different form.

It reports only on code paths exercised while the agent is attached.

By your test suite.

By QA.

By a DAST scan running against the same instrumented build.

An authorization check that exists, but is never hit with a cross-account request during the instrumented session, produces no finding either way.

IAST removes some blindness about reachability. It does not generate the traffic that would trigger the flaw.

Somebody has to write that test.

On purpose.

With a second account.

No tool category here does it for you by default.

3. “But Our Maturity Assessment Scored Us Well.”

Congratulations on the practices.

Which is what the score measured.

BSIMM is descriptive: it compares your observed activities against those of more than 100 participating companies.

OWASP SAMM is prescriptive: it scores fifteen practices across five business functions against your own prior state.

Both are useful.

Both answer the question “is the programme built properly?”

Neither answers “did this release ship with an exploitable flaw?”

A maturity model can confirm that you run SAST on merge, that you have a DAST stage, and that findings are triaged.

It cannot see which workflows the DAST stage never reached this week.

It cannot see the authorization check that compares the wrong field.

It was never designed to.

The stakes for mixing those up are not abstract.

Verizon’s 2025 Data Breach Investigations Report found exploitation of vulnerabilities was the initial attack vector in 20% of breaches, up 34% year over year.

The increase was driven substantially by zero-day exploitation of perimeter devices and VPNs.

A different estate from your application, and the same lesson.

The direction is the point.

Flaws that exist get found.

The only open question is who finds them first.

A maturity score says the practice exists. It is not evidence that the practice caught this release’s flaw.

4. “But We Have a Secrets Manager, So There Are No Hardcoded Secrets.”

You have a secrets manager.

Whether your code uses it everywhere is a separate measurement.

GitGuardian’s State of Secrets Sprawl 2025 report scanned roughly 69.6 million public GitHub repositories.

Among repositories actively using a secrets manager, the exact control meant to prevent this, 5.1% still leaked a plaintext secret in 2024.

Adoption of the tool.

Correct use of the tool.

Two numbers.

“We have a secrets manager” is an adoption claim.

“No hardcoded secrets” is a property claim.

The first does not entail the second any more than owning a seatbelt entails wearing it.

A secrets scanner makes this one of the more measurable parts of the development process, because its output is countable.

How many secrets it finds.

How many get remediated.

How long remediation takes.

That is three numbers, not a checkbox.

A scanner that finds a secret and a process that never rotates it has produced a finding, not a fix.

Tool adoption is a count of installations. The property is what the code actually does.

5. “But a Design Review Would Only Slow the Release Down.”

Then consider what the scanners cannot find even in principle.

A design flaw is not a bug in the code as written.

The code correctly implements a flawed design.

An authentication step that can be skipped by calling a different endpoint.

A trust assumption between two services that turns out to be false.

Run SAST against that implementation.

Nothing is wrong relative to its own specification, so nothing gets flagged.

Run DAST against it.

The scanner has no specification of what should have been impossible, so the skipped step looks like a working route.

Zero findings, and the flaw is load-bearing.

Design-time threat modelling is the only point in the sequence positioned to catch that population at all.

That is a structural argument, not only an economic one.

The economic one exists too.

NIST’s 2002 study of software testing economics compiled two independent industry studies against a requirements-stage baseline.

In the Baziuk (1995) data, a defect caught during system testing cost roughly 90 times the baseline.

Caught during acceptance testing, 440 times.

Caught after release, in operation and maintenance, 470 to 880 times, and as high as roughly 2,900 times where the fix needed a formal engineering change order.

Boehm (1976), on a different baseline unit, showed the same shape: roughly 5 times at system testing, 15 times at installation testing.

Those are general software defects, and the numbers are old.

The mechanism is not.

Everything built on top of an unfixed flaw, deployments, integrations, other people’s code, is part of what the later fix costs.

The design review does not slow the release down.

It moves a cost you were going to pay anyway to the only place it is cheap, and to the only place a scanner-invisible flaw can be seen.

You do not need one for every change.

You need one for changes that move a trust boundary.

6. “But the Vendor Says Its False-Positive Rate Is Low.”

Ask for the method.

Then ask for the triage hours.

Legacy static analysis tools have been documented with false-positive rates as high as 50 to 80%.

That figure describes older tooling, not every SAST product, and it should be read that way.

At that range, most of a raw report can need investigation before anyone learns whether any single finding is real.

A raw output list at that rate is not a prioritised backlog. It is a filtering problem wearing a backlog’s clothes.

Every false positive costs analyst time whether or not it turns out to matter.

That makes triage labour, not the licence, the dominant cost of running the tool.

Licensing is priced on scale, and the dimension varies by vendor.

Per developer seat.

Per application or repository.

By consumption: lines of code analysed, or scans executed.

Those grow independently, so the effective price per finding varies widely between organisations on the same nominal tier.

Two tools at the same licence price can carry very different total costs of ownership.

The difference shows up as headcount spent triaging, not as anything on an invoice.

And noise has a second cost that bears directly on the zero.

A team drowning in false positives stops trusting any single finding enough to act on it quickly.

When that team later sees a quiet report, it has two readings available.

The tool got more accurate.

Or fewer rules are firing on fewer paths.

The count alone cannot tell you which.

A low number is only good news when you know what produced it.

7. No, This Does Not Mean Scanners Are Worthless.

They are not.

Scanners catch real, pattern-shaped flaws cheaply.

Injection, deserialization and SSRF produce syntactic patterns a tool can be built to recognise, and a tool that recognises them on every merge is doing valuable work.

Keep running them.

Nor does it mean every release needs a manual review of everything.

That would be its own kind of theatre, and nobody would finish it.

The claim is narrower and harder to argue with.

A clean report is only as strong as the coverage behind it.

Authorization logic and design flaws are not what these tools were built to see.

They need tests and reviews built for them.

8. What You Should Actually Do.

Report DAST crawl coverage beside the findings count. Name the critical workflows, and fail the gate when the scan did not reach them.

A zero from a scan that missed your payment flow is a missing result, not a pass.

Configure scanners for your authenticated reality. Credentials for every role, and the multi-step flows, CSRF tokens and MFA steps a real user walks through.

Write explicit cross-account authorization tests. An authenticated second account attempting another account’s resources.

No scanner generates these by default, and IAST only reports what those tests exercise.

Hold a design-time threat-modelling step for changes that alter trust boundaries. New services, new trust relationships between services, new authentication paths.

That is where the scanner-invisible population is born.

Measure secrets handling as detection rate, remediation rate and time to remediation. Not as “we have a secrets manager.”

Track triage hours per tool when comparing scanner cost. The licence is the smaller number.

Read maturity scores as evidence about the programme. Not as evidence about the release.

9. The Actual Model Is Not Complicated.

A findings count measures what a tool examined and recognised.

It does not measure what exists.

SAST sees code without runtime context.

DAST sees runtime behaviour only where it reached.

IAST sees only what someone exercised.

None of them sees a design that is wrong in exactly the way it was specified.

Coverage percentages, false-positive rates and secrets statistics are dated evidence.

The boundary is not.

An empty report is a claim about coverage before it is a claim about safety.

So make the gate ask both questions.

What did we find?

And what did we actually look at?

If you can only answer the first, you have not shipped a clean release.

You have shipped an unexamined one with a green badge on it.