Modeling the Application Is Not Modeling the Build Pipeline

“We threat-modeled the application, so the design risks are covered.”

Which application?

The one on the diagram?

Good.

Now point at the box on that diagram that installs your dependencies.

The box that runs a script somebody else wrote, with your cloud keys in its environment.

The box where a developer committed a credential in 2022 and nobody rotated it.

There isn’t one.

Modeling the application is not modeling the build pipeline.

A threat model covers what it draws.

STRIDE walks every element on the diagram.

PASTA walks every element on the diagram.

Neither walks the elements you left off it, and neither tells you that you left them off.

That is the uncomfortable part.

A scope drawn too narrow does not fail loudly.

It finishes on schedule.

It looks thorough inside its boundary.

It produces a clean artifact, a list of threats, perhaps a pleasing number of tickets.

And a threat living outside that boundary, or at the seam between two things each modeled neatly on its own, is invisible to it by construction.

Not because anyone made a mistake inside the model.

Because the model was never asked.

Here are three well-documented ways a build went wrong without anybody attacking the application at all.

A package resolver that picked the wrong registry.

A third-party CI script that was quietly rewritten.

A secret sitting in a repository.

None of them appears on a runtime data-flow diagram.

All of them reached production credentials or production builds.

1. “But the Build System Is Not Part of the Product.”

It is part of how the product comes to exist.

That’s close enough for an attacker.

The build pipeline resolves names into code, runs that code with privileges, and holds the credentials needed to publish, deploy and talk to cloud services.

If it is compromised, the attacker changes what the build does without touching your application’s source repository.

The name for that is poisoned pipeline execution.

Your application-level threat model will not find it, because its trust boundaries sit around request handling, data stores and user roles.

It has nothing to say about a runner.

It has nothing to say about a registry.

It has nothing to say about a shell script fetched and executed on every commit.

This is not a criticism of STRIDE.

STRIDE guarantees coverage of what you drew.

That is exactly its job, and it does it.

The failure is upstream of the method.

It is the scope decision, and in most reviews nobody ever makes it explicitly.

The diagram grows around the running service because that is what the team thinks of as “the system”.

The pipeline is treated as plumbing.

Plumbing with write access to production is not plumbing.

2. “But We Use a Private Package Registry.”

Excellent.

You now meet the precondition for dependency confusion.

The attack works like this.

Your organization has internally named packages, published only to your internal registry.

Several major package managers, when a name exists in more than one place, install whichever version carries the higher version number, regardless of which registry it came from.

An attacker publishes a public package under your internal name, with a high version number.

Your build installs it.

Inside your network.

With your build’s privileges.

In 2021 Alex Birsan did this systematically as research.

He scanned for places internal package names had leaked, published proof-of-concept packages that phoned home on installation, and reached over 35 organizations.

Apple, Microsoft, PayPal, Netflix, Shopify, Yelp and Uber were among them.

Roughly 75% of the successful installs came through npm.

The bounties were paid at each program’s maximum.

$30,000 each from Apple, PayPal and Shopify.

$40,000 from Microsoft’s Azure program.

That is 3 × $30,000 + $40,000 = $130,000 across four companies, for one technique.

These companies were not paying for a curiosity.

They were assessing a path to full internal build compromise.

Note what the private registry did not do.

It stored your internal packages.

It did not tell the resolver to trust only itself for those names.

Having an internal registry is a storage decision. Resolution order is a security decision.

The control is resolver configuration.

Internally named packages resolve only from, or preferentially from, the internal registry.

That configuration lives in the build.

Which is precisely the thing your application diagram does not show.

3. “But CI Secrets Are Masked in the Logs.”

Masking decides what a job prints.

It does not decide what code running inside the job can read.

Or where that code can send it.

Codecov is the case to keep on hand.

An error in Codecov’s Docker image creation process exposed a Google Cloud Storage key.

An attacker used that key to modify Codecov’s Bash Uploader.

This is a script thousands of customers’ CI pipelines executed directly.

The altered version sent the contents of the CI environment to an attacker-controlled server.

On every run.

The CI environment is where the credentials, tokens and keys live that any pipeline step needs in order to do its work.

The tampered script ran between 31 January and 1 April 2021.

Codecov did not catch it.

A customer did, by noticing that the script’s checksum did not match.

Codecov notified customers publicly on 15 April 2021.

Log masking played no part in any of this.

The script never needed to print a secret.

It read the environment and posted it elsewhere.

Look at what the incident actually combined.

A pipeline integrity failure: the CI system trusted a third-party script by default.

A secrets handling failure: every secret the pipeline held was available to that script.

And the entry point was neither the customers’ applications nor their source repositories.

It was Codecov’s own supply chain.

One control would have addressed both halves.

Verify what the third-party script is, by checksum, before the pipeline runs it.

The customer who found the compromise was, in effect, running that control.

4. “But STRIDE Covered Every Element on the Diagram.”

Yes.

That sentence is accurate.

It is also the problem.

STRIDE and PASTA guarantee coverage of the elements on the diagram, not coverage of every attack technique.

A model can have every threat converted to a tracked item.

It can have a review date from last week.

It can still have missed an entire class of threat that was never in scope.

Internal completeness cannot detect that.

Only an external reference can.

CAPEC, MITRE’s Common Attack Pattern Enumeration and Classification, was established by the US Department of Homeland Security in 2007.

It is a public catalog of named attack patterns.

Filter it to the patterns relevant to your stack.

Spot-check them against your threat list.

Walking the whole catalog against every element would cost more than the modeling it is checking, so don’t.

Sample it.

OWASP ASVS does a related but distinct job.

Its requirements carry identifiers of the form <chapter>.<section>.<requirement>, for example v5.0.0-1.2.5 for OS command injection protection.

Mapping your derived requirements against that list finds controls your model never generated a threat for.

Usually because nobody drew the element that would have triggered them.

CAPEC asks whether known attack techniques are covered.

ASVS asks whether known control requirements are covered.

They are not substitutes for each other.

And pay attention to how the gaps cluster.

One missing pattern can be an oversight.

A cluster of missing patterns in a single attack family usually means the diagram never represented the element that family targets.

If your supply-chain patterns all come back blank, you have not found a clean pipeline.

You have found that the pipeline is not on the diagram.

5. “But Our Pipeline Tools Come From Reputable Vendors.”

So did the Bash Uploader.

Reputation is a statement about a vendor’s past.

Your CI system executes the vendor’s present.

Codecov’s customers did nothing exotic.

They ran a widely used script, supplied by the vendor, in the way the vendor intended.

The vendor’s own image-build process leaked a storage key, and that key was enough to change what thousands of pipelines executed.

The trust you place in a vendor includes the vendor’s own build.

That dependency does not appear on your application diagram either.

It barely appears on the vendor’s.

Hardcoded secrets are the same shape, closer to home.

Uber’s 2016 breach did not require a flaw in Uber’s production application.

Attackers used employee credentials, obtained through credential stuffing, to reach a private GitHub repository.

In its files were hardcoded AWS datastore credentials.

Those opened over 100 S3 buckets holding data on roughly 57 million riders and 600,000 drivers.

A private repository on a reputable platform.

A reputable cloud provider.

Neither reputation governed who could read the file.

The scale is not anecdotal.

GitGuardian’s State of Secrets Sprawl 2025 report counted 23,770,171 new hardcoded secrets added to public GitHub repositories in 2024.

That is a 25% increase on the year before.

At least 4.61% of the roughly 69.6 million public repositories scanned held at least one.

That works out to about 3.2 million repositories: 0.0461 × 69.6 million.

And 70% of the secrets leaked in 2022 were still valid when the 2025 report was published.

Found is not fixed.

A secret deleted from a file and never rotated is still a working credential.

The commit that removed it just made it harder to notice.

6. “But Scanning Every Commit Costs a Licence per Developer.”

Some of it does.

Scanning tools in CI are commonly licensed per contributing developer.

Snyk’s published pricing is a representative example of the structure.

A Free tier at $0, with limited test volume.

A Team tier from $25 per contributing developer per month.

An Ignite tier from $1,260 per contributing developer per year.

A “contributing developer” is anyone who has committed to a monitored private repository in the trailing 90 days.

So the bill tracks active headcount, not code size.

Take an invented team of 100 contributing developers on the Team tier.

100 × $25 × 12 = $30,000 a year.

A real number to put in a budget.

Now look at which controls in this piece need a licence at all.

Scoping internal package names to the internal registry is configuration.

Verifying a third-party script’s checksum before execution is a line in the pipeline.

Rotating a leaked credential is an operational act.

Drawing the pipeline as its own diagram is an afternoon of the people who already run it.

The expensive part is not the tooling. It is never having drawn the thing the tooling protects.

And one comparison is worth holding onto.

Microsoft paid $40,000 to be told about a dependency-confusion path into its own builds.

That is more than a year of Team-tier scanning for the invented 100-developer team above.

The two numbers measure different things.

One is what a finding was worth to a company that received it.

The other is what a licence costs.

Neither is the cost of the incident that didn’t happen.

But the bounty is a plain statement of how seriously the people running those builds took an unmodeled resolver.

7. No, This Does Not Mean One Threat Model Must Cover the Whole Toolchain.

It doesn’t.

A scope drawn too wide fails in its own way.

Try to model an entire multi-team platform, runners and registries and vendors included, in one workshop, and the diagram becomes too large for the room to finish or agree on.

An unfinished model protects nothing.

It does, however, look like a completed exercise on a compliance checklist.

Too wide and too narrow both produce an artifact that looks complete.

That is what makes both hard to catch.

The fix is not a bigger diagram.

It is a second one.

The pipeline gets its own model, or at minimum an explicit check against an external catalog.

And when an application model leaves the pipeline out, that exclusion is written down, with a pointer to where the pipeline is covered.

A scope decision is fine. A scope accident is the failure.

8. What You Should Actually Do

Draw the build and release pipeline as its own diagram. Include each package registry and the order in which the resolver consults them.

Include every third-party script or action executed in CI, with where it is fetched from.

Include where secrets are injected, and which steps can read them.

Then run STRIDE against those elements, like any other.

Configure package managers so internally named packages resolve only from the internal registry. Or at least preferentially from it.

Don’t leave resolution to whichever source publishes the higher version number.

Verify checksums of third-party scripts before CI runs them. A script fetched and executed without verification is a trust decision taken on every build, by default, by nobody.

Scan commits for secrets, and rotate any credential found. Deleting it from the file is not remediation.

The credential is still valid until it is rotated.

Spot-check each model against the CAPEC patterns relevant to your stack. Map derived requirements against ASVS.

Treat a cluster of blanks in one attack family as a scope finding, not a clean bill of health.

Record scope exclusions explicitly. “Build pipeline out of scope, covered by pipeline model, last reviewed on this date” is a decision.

Silence is not.

9. The Actual Model Is Not Complicated.

A threat model answers: what can go wrong with the things we drew?

It cannot answer: what did we not draw?

That second question is the scope decision, and it has to be asked on purpose.

The application runs in production.

The pipeline decides what runs in production.

Both are systems. Both need a diagram.

Package names, script checksums and secrets counts will change.

The tooling prices will change.

The structure won’t.

A model covers what it draws.

Draw the pipeline.