The Old Problem Was Hard Enough
For the better part of a decade, “software supply chain security” meant one thing: tracking what’s inside your code. Which open-source packages. Which versions. Which transitive dependencies three layers deep that nobody consciously chose.
SolarWinds, Log4Shell, and XZ Utils all taught the same bitter lesson. The risk lives less in the code a team writes itself, and more in the millions of lines they pull in from strangers on the internet. Locking down that pipeline became an industry obsession.
Then AI joined the build. And the question got a lot messier.
AI Doesn’t Just Help You Code — It Becomes a Dependency
When a developer pastes a prompt into GitHub Copilot or OpenAI‘s ChatGPT and accepts a suggestion, that snippet enters the codebase like any other contribution. But it’s fundamentally different from a line written by a human teammate.
You didn’t interview the model. It doesn’t have a security clearance. It doesn’t know your threat model. And critically, you have almost no visibility into the data it was trained on — which means you can’t audit whether that suggested function contains logic lifted from a GPL-licensed project, or worse, a known vulnerability.
This is not theoretical. Studies have shown that AI code assistants can regurgitate verbatim blocks from copyrighted repositories. They also hallucinate package names, suggesting libraries that don’t exist — a perfect vector for dependency confusion attacks.
The Provenance Problem Gets Worse
Traditional supply chain security relies on provenance. You know where a package came from because it has a signed manifest, a checksum, a chain of custody. With AI-generated code, the lineage is opaque. The model is a black box. You can’t ask it, “Where did you learn this algorithm?” and get a verifiable answer.
Some teams are already experimenting with adding AI model identifiers to their software bill of materials (SBOM). But that’s a nascent practice. Most organizations don’t even have a complete SBOM for their human-written dependencies, let alone one that accounts for AI contributions.
Three New Risks You Can’t Ignore
Adding AI to the build pipeline introduces categories of risk that traditional supply chain tools weren’t designed to catch.
- Invisible backdoors: A model trained on public code may have absorbed intentionally planted vulnerabilities. Researchers have demonstrated “poisoned” training data that causes models to suggest insecure code patterns when triggered by specific prompts. You wouldn’t know unless you tested every suggestion against a known exploit database.
- License contamination: AI models don’t respect licenses. They train on everything. If a model suggests code that’s functionally identical to a GPL-licensed function, your proprietary project could suddenly have a compliance problem. The legal landscape here is unsettled, but the risk is real.
- Loss of institutional knowledge: When a human writes a tricky piece of logic, they usually understand why it works. When AI writes it, the developer accepting the suggestion may not fully grasp the implications. The code passes review, but the reasoning behind it doesn’t. That’s a maintenance and security time bomb.
What Security Teams Should Do Right Now
You can’t ban AI code generation — it’s already too widespread. But you can adapt your supply chain practices to account for it.
First, treat every AI-generated code block as a high-risk third-party dependency. Subject it to the same scrutiny you’d give a new open-source library. Run static analysis. Check for known vulnerability patterns. Review it line by line, not just glance at the diff.
Second, update your software supply chain security policy to explicitly address AI contributions. Require that developers tag or comment code that was generated by an AI model. This makes it auditable. It also helps your security team spot patterns — if a particular model keeps suggesting vulnerable code, you can flag it.
Third, push your tooling vendors for transparency. Ask your AI coding assistant provider what data their model was trained on. Ask if they test for vulnerability injection. Demand a model-level SBOM. The market is young enough that customer pressure can shape the roadmap.
Accountability Is the Hardest Question
In traditional software supply chains, there’s a clear chain of accountability. If a package has a vulnerability, the maintainer is responsible. If a developer introduces a bug, the code review process catches it — or the developer owns the fix.
Who owns the bug when AI wrote the code? The developer who accepted the suggestion? The team that trained the model? The vendor who sold the tool? The legal and engineering answers are still being written. But the prudent approach is to assume that accountability ultimately rests with the human who committed the code. That means your review process needs to be robust enough to catch AI-generated mistakes before they reach production.
The software supply chain was already fragile. Adding AI to the mix doesn’t break it — but it does expose new weak points. The teams that acknowledge those weak points early, and build processes around them, will be the ones that don’t get burned when the next SolarWinds-style incident turns out to have an AI origin story.
