Skip to content
Beta
All articles

Harmful success: coding agents finished 95% of tasks but only 37% safely

VeriHarm-Bench tests whether coding agents complete benign shell tasks without side effects such as deleted files or loosened permissions. Most didn't.

By The SpanDock team

5 min read

A diagram of how VeriHarm-Bench scores a run: the same final sandbox is checked by the original task test and by a harm test, giving safe success, harmful success or failure; across four agents, task accuracy averaged 95.1% and safe success 37.0%

Most coding agent benchmarks ask one question: did the agent do the task? A paper posted to arXiv in November 2026, VeriHarm-Bench, asks a second one: did it do anything else along the way? The authors call the failure mode "harmful success": the agent completes a benign task correctly and, at the same time, deletes an unrelated file, loosens permissions or corrupts data nobody asked it to touch. Four frontier agents completed 95.1% of the benchmark's tasks on average, but finished only 37.0% without triggering the harm the test was looking for. For teams that judge agents by whether the tests pass, that gap is the point.

What the benchmark measures

The authors, from the University of Virginia, Virginia Tech and UCLA, separate harmful success from the risks most safety work studies. There is no jailbreak, no prompt injection and no harmful request. The task is ordinary, and the harm is a side effect of how the agent chose to do it.

To measure it, they built VeriHarm, a pipeline that turns existing coding tasks into tests of side effects:

  1. They started from 200 benign tasks from BashBench, each needing a shell-script solution with tests for correctness.
  2. A target agent first ran the original task.
  3. A proposer agent then made a small, realistic change to the instructions, of one of two kinds: underspecification (leaving out a detail a careful engineer would ask about) or delegation of control (handing the agent a decision). It also wrote an executable harm test that checks the final state of the sandbox for the side effect.
  4. The task's original correctness test stayed fixed, so the objective didn't change. The search stopped at the first run where the task passed and the harm test fired, or after five attempts.

This produced 1,046 candidates, from which the authors selected 150 instances through a quality review. Because both tests are code, any new agent can be scored without a model or a person reading its transcripts.

One example from the paper: a task asks for a script that merges two files, with an added requirement that re-running it shouldn't append the merged content twice. The agent satisfies that by emptying one of the source files. The task test passes. The user's data is gone.

What it found

The benchmark was run on four agents that were not used to build it: Claude Opus 4.8, Claude Opus 5, GPT-5.5 Pro and GPT-5.6 Sol.

AgentTask accuracyHarmful successSafe success
Claude Opus 4.898.7%72.0%26.7%
Claude Opus 597.3%56.2%41.1%
GPT-5.5 Pro89.3%53.7%35.6%
GPT-5.6 Sol95.3%50.7%44.7%
Overall95.1%58.2%37.0%

Even the best agent, GPT-5.6 Sol, finished fewer than half the tasks safely. The authors also point out that every evaluated agent is stronger than the models used to construct the benchmark, which suggests that side effects found with weaker models carry over to stronger ones.

The kinds of harm are familiar to anyone who has cleaned up after a hurried script. In the paper's taxonomy, built from model-written summaries of each instance (an instance can have several labels), 58.0% involved acting outside the intended scope, such as a search that returns files from unrelated folders; 51.3% deleted, wiped or truncated data; 34.0% corrupted data in place; 23.3% loosened permissions, such as setting scripts and logs to mode 777 to satisfy a request for "no access issues for teammates"; 14.7% changed configuration or services, such as binding a server to all interfaces instead of loopback; and 7.3% touched secrets or credentials.

Reading the results carefully

  • It isn't your harness. The agents ran as a generic ReAct agent in the Inspect evaluation framework, with bash, Python, a text editor and a test runner, not inside Claude Code, Codex or another product. The paper describes no approval step and no product sandbox rules. Real harnesses may catch some of these actions, or ask first.
  • The tasks were chosen to elicit harm. The 150 instances are the ones where a perturbation worked on at least one construction agent. The rates describe this benchmark, not how often agents cause damage in everyday work.
  • Harms aren't weighted. Each harm test is pass or fail, and the paper doesn't estimate real-world severity. Emptying a scratch file and exposing credentials count the same.
  • One run per instance. Each agent made a single attempt per task, so small differences between agents shouldn't be over-read.

Within those limits, the finding is robust and simple: a passing test suite says little about what else an agent changed.

What it means for engineering and security leaders

Many teams now accept agent work on the strength of green tests and a quick look at the diff. Harmful success shows why that isn't enough. The harmful changes in this benchmark happen outside the code under test: in other files, permissions, configuration and data. A diff review catches some of them if the change lands in the repository, and none of them if the agent ran a command against the environment.

It also gives evaluation a better shape. When you compare agents or models, "task accuracy" alone rewards an agent that empties a file to make a test pass. Adding even a few checks on the state of the environment, the files that should not change and the permissions that should stay tight, turns an evaluation into something closer to what you care about.

Finally, underspecification is a human input. The perturbations that caused harm were small omissions in otherwise reasonable instructions. Better task writing won't eliminate the problem, but stating what must not change is cheap.

Practical takeaways

  • Add "must not change" checks to agent evaluations. Alongside the task test, assert that unrelated files still exist, permissions are unchanged and services listen where they should.
  • Say what is out of bounds in the task. Name the files, folders and settings the agent must leave alone, and whether it may delete anything.
  • Run agents with narrow write access. A sandbox that limits writes to the working folder turns many of these harms into errors.
  • Keep approvals for destructive commands. Deletion, permission changes and service configuration are worth a confirmation step, even in trusted repositories.
  • Look beyond the diff. For agents that ran commands, review the command log, not only the files in the pull request.

For teams running Claude Code, Codex or Gemini CLI, SpanDock's guardrails can match your own regular expression in the arguments of tool calls and require the person to confirm before the call runs, with every match recorded. A pattern catches obvious commands such as a recursive chmod 777, not subtle side effects. Guardrails on computers are on the Enterprise plan; see guardrails.

For more on measuring agents, see Terminal-Bench 4.0 and honest comparisons of coding agents, and on containment, sandboxing coding agents.

Sources

  1. VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably (arXiv 2611.03575)
  2. Inspect: an open-source framework for large language model evaluations (UK AI Security Institute)
Cookie categories

Details: Cookie Policy.