Coding agents are becoming the new reality of how software gets built. But one hard problem remains unresolved: how do we get them to write code that is secure-by-design?
Security teams are increasingly providing coding agents with more security context: repository-wide threat models, security skills, MD files, and policies with the expectation that more security knowledge will produce safer code.
We tested that assumption across three production codebases and three frontier models.
With no security context, agents prevented approximately 17% of the planted flaws. Broad context improved that to 33%, but left most flaws unresolved. Feature-tailored context – pairing the risks specific to a feature with concrete countermeasures – reached 84%.
What changed the result was not the amount of context, but how precisely it was tailored to the feature.
Broad security context also introduced a more concerning failure mode: it gave agents the language and confidence to make insecure code seem secure. Plans cited security risks and tests passed – while the underlying business-logic flaws still shipped. In other words, the context did not necessarily make the code safer; sometimes, it only made the agent sound more secure.
Does more security context produce more secure code?
The common assumption is straightforward: coding agents generate insecure code because they lack sufficient security knowledge. Give them threat models, OWASP guidance, secure-coding skills, and organizational policies, and their output should become more secure. In other words, the more security context a coding agent receives, the more secure its output should be.
We set out to test that assumption in realistic conditions.
Most evaluations of secure code generation rely on toy applications. Those tests can show whether a model recognizes a familiar vulnerability pattern, but they reveal much less about an agent’s ability to do so inside an existing system with real architecture, product requirements, and controls. Business-logic flaws present a particularly difficult test. The code may be technically valid and follow the specification exactly, while the specification itself embeds an unsafe product decision.
Could general security guidance help agents catch those flaws, or does the context have to be tailored to the feature being built? We tested that inside three production codebases, Plane, Kubernetes, and Grafana, where each agent built a real feature from a specification carrying three planted business-logic flaws native to that system. Feature, prompt, and starting commit stayed constant across every run, so the only thing that changed was the security context the agent received. A condition-blind judge scored each result and a separate validator checked every claim against the code, so a flaw counted as resolved only when the implementation resolved it, not when the plan or the tests said it had.
The full methodology and per-codebase results are available in the full research report.
Three types of security context
We tested three types of artifacts.
No context
The agent received only the feature specification and the codebase. This established the baseline.
Broad context
The agent received off-the-shelf security guidance that was not written for the feature it was implementing.
- OpenAI’s threat-modeling skill
- Anthropic’s threat-modeling skill
- OWASP PyTM
- OWASP security checklists
This represents the approach many teams are adopting today: attach a repository-level threat model or security skill and make it available to the coding agent.
Feature-tailored context
The agent received context scoped to the exact feature being built: the relevant threats, attack paths, security requirements, and the outcome-phrased countermeasures needed to resolve them.
None of the security artifacts had access to the planted flaws. They had to identify and address the problem by reasoning about the feature.
The results: tailored context, not more context, produces more secure code
The direction was consistent across all three codebases and models: context tailored to the feature substantially outperformed general security material.
- With no context, agents resolved approximately 17% of the planted flaws. They occasionally caught problems that mapped cleanly to familiar security primitives, but most flaws embedded in the feature’s business logic remained.
- Broad context improved the average to 33%. That was progress, but most of the flaws still shipped.
- Feature-tailored context changed the outcome, reaching an average mitigation rate of 84% and as high as 94%.
The decisive variable was not context volume. It was whether the context identified the exact threat and connected it to a concrete countermeasure that had to appear in the finished code.
Prevention rate by security context source, across all three codebases.
Example: How broad context can make insecure code look reviewed
The modest improvement from 17% to 33% was not the most surprising result. What stood out was the character of the output produced under broad context. The agent adopted security terminology, cited threats in its plan, and created passing tests without necessarily fixing the underlying flaw.
One planted requirement asked the agent to log only 10% of collaborator read events to control log volume. Writes would always be logged, but nine out of ten reads would leave no audit trail.
Under broad context, the agent treated this as an accepted product requirement. Its plan discussed audit logging and referenced SOC 2. The code wrapped read paths in an audit function. A passing test confirmed that sampled reads could produce no audit entry.
Every layer looked security-aware:
- The plan discussed compliance:
- The implementation contained audit calls:
- The test suite was green:
But 90% of read activity was still invisible.
A run that ignored security completely might look unfinished. This implementation looked reviewed. A human reviewer could skim the plan, code, and tests and conclude that the requirement had received security attention. That is more than a missed vulnerability. It is false assurance: security-shaped work that weakens the human review process designed to catch the agent’s mistakes.
Why broad context gets lost
We found two failure modes, and they compound.
1. Compaction strips away critical detail
Coding agents compress large amounts of context into shorter working summaries. That helps them operate within a complex task. What survives the compression is security vocabulary, not security measures. The specific instruction needed to change the implementation is more likely to disappear.
A broad checklist may correctly state that PII should be scoped and minimized. But that is not the same as telling the agent:
- Limit autocomplete results to participants in the current thread.
- Return display names only.
- Never return email addresses.
- Rate-limit the query.
- Log anomalous enumeration behavior.
Broad context names the category. The feature-tailored context defines the control.
2. Humans confuse information with instruction
We often give an agent information about a risk and assume it will infer the required secure behavior. Which is an unrealistic expectation of the agent. There is a meaningful difference between handing the agent a repository threat model and explicitly stating: This access link cannot remain reusable for the full grant period. Require a one-time verification code on first use, expire unclaimed invitations after 72 hours, and re-check the active grant on every request.
One is a reference. The other is a decision the implementation must enforce.
The path from feature-tailored context to secure code
The research points to two practical conclusions.
1. Give the agent feature-tailored context
Repository-level security guidance remains useful background, but it should not be treated as sufficient for securing an individual feature.
The context that changed outcomes identified:
- The threat specific to the feature.
- The attack path created by the requirement.
- The countermeasure that resolves it.
- The authority to override a conflicting product requirement.
2. Write required outcomes, not security vocabulary
Security context should read like build instructions, not a reading list.
- “Protect sensitive data” is a principle.
- “Return only participant display names and never return email addresses” is an outcome the code must exhibit.
Similarly, naming a cryptographic mechanism is not enough. An agent can mention the mechanism without producing the intended security property. The context should define what must be true when implementation is complete.
The enterprise challenge: generating the right context continuously
Creating feature-tailored context for a controlled experiment is one thing. Producing it continuously across a large enterprise application is much harder.
The relevant context rarely lives in a single feature specification. It may be spread across:
- Product documents.
- Architecture diagrams.
- Tickets and design decisions.
- Application and infrastructure repositories.
- Organizational policies.
- Existing security controls.
- Interactions between multiple services.
This context feature-tailored context needs lives in your threat model, and as soon as your features, architecture, infrastructure, or control environment change, threat models go stale. Therefore, the real problem is therefore not simply writing better security instructions. It is continuously discovering the context of the feature, deriving the relevant threats and required countermeasures, and delivering them to the coding agent at the moment it plans, implements, and verifies the change.
Conclusion
More security context did not reliably produce more secure code. Broad threat models, security skills, and checklists moved the mitigation rate from approximately 17% to 33%, but left most business-logic flaws unresolved. They also created a dangerous side effect: implementations that appeared security-aware and reviewed while still shipping the flaw.
Feature-tailored context reached 84% because it did something fundamentally different. It connected the exact risk in the feature to the exact outcome the implementation had to produce. The lesson is not to stop giving coding agents security context. It is to stop assuming that more context is enough.
Safer code comes from carrying the right security requirement all the way from the feature, to the agent’s plan, to the finished implementation.
