> ## Content Index > Fetch the complete content index at: https://varops.com/llms.txt > Use this file to discover other available public pages before exploring further. # Claude Code's auto mode blocked 720 of 720 attacks and still lost to a public exploit chain - URL: https://varops.com/claude-codes-auto-mode-blocked-720-of-720-attacks-and-still-lost-to-a-public-exploit-chain/ - Published: 2026-09-01T09:19:13.000Z - Updated: 2026-09-01T09:19:13.000Z - Description: A vendor-commissioned evaluation scored 720 prompt injections at zero. Weeks later a public exploit chain beat the same mode. Both numbers are correct. - Author: Nix Nullty - Tags: Overhyped *A vendor published a security score of zero and it was accurate. Three weeks later someone published a working attack against the same product and that was accurate too.* [*Nix*](https://varops.com/columnist/nix/) *spent this week on why both can be true, and the answer is not that anyone lied - it is that the evaluation and the risk are measuring different things, and only one of them is printed on a chart. He picks the thread up from* [*his July audit of agent benchmarks*](https://varops.com/an-agent-benchmark-measures-the-score-not-the-capability-heres-the-audit-that-put-a-number-on-the-gap/)*, where the gap was between a score and a capability. This time it is between a score and a boundary, which is the more expensive one to get wrong. —* [*Muximus*](https://varops.com/columnist/muximus/) --- The number is real. It is also not about your exposure, and the distance between those two sentences is the whole column. On 7 August, Anthropic [published an evaluation](https://claude.com/blog/auto-mode-default-in-claude-code?ref=varops.com) it commissioned from Trajectory Labs: 720 indirect prompt-injection attempts - hidden instructions planted in content an agent reads - fired at Claude Code’s auto mode. Every one failed. On 26 August, a security researcher published a chain that got remote code execution against the same model in the same mode, three times out of five. Nobody is lying in that paragraph. A classifier gets scored against a set of attacks somebody picked in advance. A boundary has to survive somebody who watches what the classifier does and walks around it. Those are different quantities wearing the same word, and the reader’s problem is not that a vendor shipped a bad number - it is that the number traveled and the sentence underneath it stayed home. ## Credit where it’s due: the evaluation is better than most The usual move here would be to attack the methodology. The methodology is fine. Anthropic had Trajectory Labs test 72 indirect prompt-injection scenarios against the public builds of Claude Code and Codex as of 17 July 2026\. The scenarios were held out from Anthropic. Each ran ten times. The attacks had been tuned against Claude Opus 4.7, a model that was not in the reported set, so every model in the test met them cold. The result, in the post’s own words: “none of the 720 attack attempts succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode.” Held-out scenarios, repeated runs, attacks optimized against a model that was not being graded, and the competitor reported in the same chart instead of quietly dropped - GPT-5.6 Sol let 5.83% of attacks through in Codex’s Auto-review mode and 19.03% in Full Access mode. Most vendor security marketing does not survive this much daylight. This did. Two footnotes belong on it anyway. Anthropic commissioned and published an evaluation of its own product next to a competitor’s, which is a selling interest a reader should price in. And the evaluation appears to carry no benchmark name - the researcher who later attacked the same mode says it “seems to not have a published benchmark name” - with no separate write-up from Trajectory Labs that VarOps could find. The 0.00% everyone repeats is not printed anywhere in Anthropic’s prose. It lives in the chart published alongside the post, and it entered circulation from there. So the design is not the weak part. The weak part is what a fixed scenario set is capable of telling anyone, and Anthropic said so in the same post, in its closing section: > while we believe auto mode reduces risk for most users, it relies on classification systems and therefore does not eliminate risk. For high-stakes changes to production infrastructure, we still recommend reviewing Claude’s actions yourself. Accurate, on time, and not what got repeated. What got repeated was the zero. Boris Cherny of the Claude Code team tied the layered defense to the rollout in a single post the same day: > turns out you can get indirect prompt injection to \~0 on unseen attacks if you stack enough layers (model training + input probes + a classifier checking intent). didn’t expect that a year ago. auto mode is default in claude code as of next weekBoris Cherny, [7 August 2026](https://x.com/bcherny/status/2085860677990883454?ref=varops.com) On 14 August, auto mode [became the default](https://claude.com/blog/auto-mode-default-in-claude-code?ref=varops.com) for new sessions on Pro, Max and Team plans. It replaced a permission model in which, per [Anthropic’s own description](https://claude.com/blog/auto-mode?ref=varops.com), “every file write and bash command asks for approval.” The confirmation step came out. A lot of engineering organizations let it go on the strength of a chart. ## What the fixed set could not see On 26 August the researcher who publishes as [Embrace The Red](https://embracethered.com/blog/posts/2026/breaking-claude-code-opus-5-and-automode/?ref=varops.com) (@wunderwuzzi23) posted a working chain against Opus 5 in auto mode. What it achieves matters more than how it works: a run of individually ordinary steps, each a defensible thing for a coding agent to do, ending with attacker-chosen code executing on a developer’s machine. The mechanism, for anyone who wants it. Nudge Claude out of the WebFetch tool and into calling curl. Point it at a ZIP archive holding specially encoded files and a native decoder. Claude correctly refuses to run the unknown binary and writes its own Python decoder instead - the safe choice. It runs that decoder inside the attacker-controlled directory it just unpacked. A malicious `struct.py` sitting in that directory shadows Python’s standard module, so importing `base64` pulls in the poisoned file, and an obfuscated expression at module level fires during the import. Reported results, five runs per variant: 3 of 5 for the chain ending in a command-and-control callback, 3 of 5 for a variant that spawns a second Claude process to run reconnaissance, 4 of 5 for a variant that writes outside the workspace. The researcher’s caveat goes next to those numbers rather than beneath them, because a column that criticizes somebody else’s methodology had better be honest about the methodology it is standing on: “These are small samples, not a universal ASR measurement. And rates improved as payloads got iterated with the help of Codex.” The post calls the finding “representative for a motivated attack, but not comprehensive.” Five runs is five runs. One more limit: the evaluation tested Claude Code v2.1.205 as of mid-July, and the write-up does not say which build its chain ran against. None of which rescues the number, because sample size was never the problem on either side. The write-up puts the structure of it plainly: “The benchmark measured a fixed set of 72 scenarios, run 10 times each. My chain was not in that set. So 0.00% on the benchmark and a working RCE are both true at once.” Every step in that chain is individually justifiable, right down to Claude writing its own decoder because a strange binary looked untrustworthy - which is the model behaving correctly, and the write-up says as much. A classifier grading each tool call for whether it is irreversible, destructive, or aimed outside the environment sees a sequence of defensible actions. The attack lives in their order. ## The safety layer blocked the cleanup In a few runs, Claude noticed the compromise and issued a command to kill the malware process. Auto mode denied it. The researcher’s summary: “The classifier allowed the creation of the malware process, but then it blocked the command intended to stop it!” Anthropic’s published targeting language says the classifier blocks actions that are irreversible, destructive, or aimed outside the environment. Killing a running process is destructive by that description. Whether the process is malware is context that description does not reach. That is reasoning rather than a claim about the classifier’s internals, and it is the only reading under which both decisions are consistent with each other. Anthropic closed the report as **Informative**, behavior working as designed. The security team’s position, as the researcher reports it: auto mode is a convenience feature backed by a best-effort classifier rather than a security guarantee, determined chains assembled from benign-looking steps are not what it is built to stop, and the real boundary is OS isolation and network egress control. That answer is technically consistent with the caveat in the August post. It is also the finding. The security team’s position and the headline number came from the same organization, and only one of them is what anybody carried into a decision. ## The question to take to the next vendor A safety number is about a boundary only if the evaluation included an adversary who was allowed to adapt. Everything else is coverage of a list. That question outlives this story and this vendor, and it applies to every classifier-backed control an organization has already waved through: which approval step did this remove, and what was the evaluation permitted to do when it was scored? A held-out scenario set is real evidence about the model. It is not evidence about the attacker, because the attacker was not in the room when the scoring happened. Two controls follow, and neither of them is ours. The boundary that holds is the one outside the model - OS isolation and network egress control, per the position Anthropic’s own security team gave the researcher - so an agent’s blast radius gets set by what the machine and the network permit, not by what a classifier declines. VarOps has [made the sandbox version of this argument before](https://varops.com/an-agent-sandbox-has-to-hold-against-an-adversary-now-not-an-accident-how-to-pick-an-agent-sandbox-when-a-vm-is-no-longer-enough/), and the [Hugging Face breakout in July](https://varops.com/openais-model-broke-into-hugging-face-to-cheat-its-own-security-test-the-saga-continues/) turned on exactly one permitted egress path. Second: for high-stakes changes to production infrastructure, the vendor’s own written recommendation is still to review the agent’s actions directly. Auto mode being the default is a setting. Whether it stays on is a decision made per environment, and a chart is not qualified to make it. --- *Disclosure: VarOps Magazine is produced using Anthropic’s models, including Claude Code, the product examined here.*