AI Security Tools Could Fuel Cyber-Attacks, Researchers Warn
Organizations are rushing to deploy AI-powered coding agents from Anthropic and OpenAI to automate vulnerability discovery and patch management. But a new report warns that the very access these tools require could turn them into powerful attack vectors.
Published July 8 by the AI Now Institute, the research demonstrates a proof-of-concept exploit that achieves remote code execution (RCE) through two of the most popular AI-powered command-line interfaces: Anthropic’s Claude Code and OpenAI’s Codex. The exploit works against Claude Code running Claude Sonnet 4.6, 5, or Opus 4.8, and Codex using GPT-5.5.
The attack is alarmingly simple: a victim can be compromised just by asking the AI to review or analyze a third-party open-source codebase — a widely recommended defensive use case.
How Prompt Injection Enables Silent Remote Code Execution
The attack begins with an attacker hiding malicious instructions inside an open-source library’s files, embedding them in code comments or documentation in a way designed to manipulate how the AI interprets commands.
The victim then uses Claude Code or Codex in “auto-mode” or “auto-review” mode — a standard feature that automatically executes commands the AI judges safe, only pausing on flagged risks. Because the injected instructions are crafted to trick the AI’s judgment, the assistant is fooled into believing the attacker’s commands are harmless or routine. It runs them automatically, without alerting the user.
Multi-Stage Injection and Tool-Use Exploitation
The key mechanism is a multi-stage prompt injection combined with tool-use exploitation. When the AI agent scans the repository, it doesn’t just read code passively — it builds a semantic model of the project by parsing source files, scripts, and documentation. The attacker exploits this by embedding natural-language instructions inside trusted-looking artifacts (e.g., README.md) that the model interprets as part of its task context rather than untrusted input.
These injected instructions reshape the agent’s planning process. Instead of directly telling the model to execute something obviously malicious — which would trigger safeguards — the instructions suggest that a specific script (e.g., security.sh) is a standard part of the project’s security workflow. They frame execution of that script as necessary to complete the user’s request (“run security checks”) and align with the agent’s goal of vulnerability analysis, making the action appear legitimate.
The repository also contains a second-stage payload: a shell script that appears to run common tools like linters or static analyzers, a hidden malicious binary that the script executes, and a decoy source file that makes the binary look benign and consistent with expected build artifacts.
When the agent evaluates whether to execute the script, it relies on its internal classifier and heuristics. Because the script references familiar security tooling, the binary appears to correspond to legitimate source code, and the documentation frames execution as routine, the agent misclassifies the action as safe. In auto-mode, this classification is critical — the agent is explicitly authorized to execute shell commands without human approval if they are deemed low-risk.
The result: the agent autonomously decides that running security.sh is part of the requested analysis, executes the script via its tool interface, indirectly launches the malicious binary, and triggers arbitrary code execution on the host system. Remote code execution is achieved — the attacker’s code runs on the victim’s machine, even though the victim believed they were having the AI passively scan a codebase for vulnerabilities.
An Attack with Low Requirements
What’s notable is how little is required to pull this off. No special hooks, plugins, skills, model context protocol (MCP) servers, or custom configuration files are needed. It works with a completely out-of-the-box install of either tool.
The victim simply needs to run the assistant in its standard automated review mode and point it at a codebase containing the attacker’s hidden instructions — something as ordinary as asking the AI to “scan this library for vulnerabilities.” The researchers tested this on Linux systems using specific versions of both tools: Claude Code versions 2.1.116, 2.1.196, 2.1.198, and 2.1.199, and Codex version 0.142.4.
The significance of this finding is that it undermines the idea that these AI agents can safely be used for defensive security work, since the attack surface is identical to the access required for their intended, legitimate purpose. The researchers emphasized this as governments and companies push to deploy these tools more broadly for automated security review and patching, including initiatives like Anthropic’s Project Glasswing, Palantir’s MA-S2 standard, and OpenAI’s Patch the Planet and Daybreak programs — some of which touch safety-critical infrastructure.
The technique could likely transfer to other agentic AI coding platforms beyond Anthropic’s and OpenAI’s, the researchers argued, because the core issue is architectural. Giving an AI agent the autonomy to decide for itself what’s safe to execute creates a new trust boundary that attackers can target directly, by convincing the AI — rather than the human — that malicious code is safe to run.
While the researchers noted that their report “is not within the scope of the security disclosure policies for either Anthropic or OpenAI,” Khlaaf and Milanov contacted both companies to inform them of their findings and offered support to verify the issues raised.
Architectural Risk Undermines AI Agent Safety, Warns Expert
Eljan Mahammadli, head of AI provenance at Polygraf AI, said the significance of the research lies in the underlying weakness it exposes, not the specific exploit used.
“An AI coding agent has no reliable way to distinguish the text it reads from instructions it is supposed to follow,” he said. That’s because everything in its context window is processed with the same authority. That lack of attribution means malicious instructions, once embedded, are treated as equally trustworthy — which is why similar attacks keep reappearing in different forms.
He argued this is not something a model update can fix, since it reflects a deeper architectural issue. “The problem is a property of how these systems handle language and not a defect that can be trained away,” he said. From a provenance perspective, he described it as a failure of attribution, where the agent cannot determine where text comes from or whether it should be trusted.
Nevertheless, Mahammadli pushed back on the idea suggested by the AI Now researchers that the findings undermine AI’s role in defensive security. He said the issue is specific to a common but flawed setup: agents that combine access to untrusted data, command execution, and sensitive environments in a single process, with only a safety classifier as a guardrail.
“When those powers sit together, a single injected instruction is enough to turn the agent against its operator,” he said, arguing that stronger runtime controls and separation of capabilities are key.
He also highlighted that, counterintuitively, more advanced models sometimes detected inconsistencies in the exploit but executed it anyway. This challenges the assumption that stronger models are inherently safer. “A more capable and more compliant agent can simply be a more effective executor of whatever instruction reaches it,” he warned, cautioning that deployment in critical systems is moving faster than solutions to this core trust problem.
For organizations already using these tools, the findings serve as a stark reminder: AI security tools can be turned against their users with surprising ease. Until the underlying architectural issues are addressed, the very features that make AI coding assistants powerful also make them dangerous.
CyberSecurity6 months ago
How To4 months ago
CyberSecurity6 months ago
Social Media6 months ago
Infosecurity6 months ago
CyberSecurity6 months ago
Video4 months ago