Done Is Not Safe: 70% of Finished Agent Runs Were Both

The agent finished the job. The benchmark finished the argument.

AgentS4D ran 6,560 risk-injected cases across four harnesses and five model backends this week, and of the runs that completed, 70.52% also earned an Unsafe verdict. Flip it round and it gets worse: of every run flagged unsafe, 97.38% delivered the assigned task anyway.

Those two numbers together end a habit. A completed task has stopped being evidence of anything except completion, and the builders worth reading this week all converged on the replacement: receipts something other than the agent can produce.

The least-upvoted important thing on the sub

The paper sat at 5 points, which says more about the sub than the paper. The authors turned 76 workspace tasks into 328 risk-injected cases and ran the full grid, and the finding held across every harness and backend combination they tried.

Credit where it lands: the authors scoped their own result, warning the figures neither isolate a causal carrier nor stand in for production incident rates. That restraint makes the core gap harder to dismiss, not easier. A team this careful about what its numbers do not say has earned trust for what they do say, and what they say is that task completion and safe execution are separate verdicts.

The cheap layer, and who skips it

A quieter thread asked who checks that the agent built the right thing, and the best answer refused to start with a human. Unit tests, contract tests, and a lint pass run before any human looks, killing most wrong-thing cases mechanically. As the commenter put it: “If a machine can check it, the machine checks it first.”

The honest limit came from the same thread: a second AI catches syntax and misses whether the whole concept is dumb. Mechanical checks bound the damage. They do not confer taste.

o8 builds the doctrine into a merge gate

The clearest product expression is o8, an MIT-licensed orchestrator that names the pain precisely: run several coding agents and your day becomes five terminals with no shared memory and git log as the only audit trail.

Its answer is architecture rather than policy. One orchestrator scopes and dispatches, every worker runs in its own git worktree, and nothing merges without approval through a gate of rebase, typecheck, head-SHA lock, and ordered merge. Workers have no path to merging their own output; in o8’s own framing, “execution and approval are separate authorities by construction”.

One commenter supplied the design law: “Letting agents merge their own work is how you get silent conflicts…” The dogfooding number closes the case: o8 dispatched more than 5,500 of its own commits through its own gate.

The memory war is the same war

The day’s loudest fights, whether to delete your CLAUDE.md and how agents should remember across sessions, look unrelated and are not. They are the receipts problem one layer up.

Twin, an open-source continuous-context project, drew the sharpest line of the week from a sceptic: “Cognitive continuity without receipts is just a nicer cache.” His standard is provenance and freshness, every synthesised claim tracing to a specific message or PR with a timestamp before anything downstream acts on it.

That standard settles the memory war without a benchmark. A vector index nobody can open, edit, or diff fails it; a plain-text note that survives git-diff passes. CLAUDE.md files rot for the same reason unverified memory is dangerous: an unaudited claim quietly promotes itself to a permanent instruction.

Also in the buffer

Boris Cherny reportedly floated deleting your CLAUDE.md in a fresh 35-minute interview, and the sub chose doc hygiene over arson. The working theory: most files are bloated with guardrails written for older, weaker models, and what compensated for a weak model is tax on a strong one. One builder nuked his on a test account and measured no quality drop.

Grep beat the graph. Two identical Claude Code subagents ran the same eight-site edit on a production TypeScript monorepo; recall tied at 8 of 8, and the knowledge-graph agent took 3m06s to grep’s 2m12s. The graph added zero accuracy at 40% more time, which is a bill, not a benchmark.

Sonnet 5 is quietly good. A builder who had not opened Sonnet in three months found a job that ate 20% of his weekly Fable limit in 20 minutes took two or three Sonnet prompts and 10 minutes. Loyalty to one model priced itself again.

The counterweight to the GTA 0.6 tech demo that grabbed the headlines: a free-to-play drag racer built almost entirely with Claude Code passed 5,000 players sixteen days after launch and booked its first real income. It is still buggy, complete with a car that makes kissing noises on every gear shift, and it shipped, and it is played.

By the numbers

70.52% of completed AgentS4D runs also earned an Unsafe verdict
97.38% of unsafe runs still completed the assigned task
6,560 total runs across four harnesses and five model backends
5,500+ commits o8 dispatched through its own merge gate
40% more time for zero added accuracy: the knowledge graph against grep
5,000 players in sixteen days for the buggy racer that shipped

The close

The teams already shipping merge gates, freshness gates, and mechanical pre-checks will catch the next AgentS4D-shaped incident before a user does. Everyone still grading agents on task completion will find out from the user, and the 97.38% figure says the task will have finished beautifully by then.