AI

你的 Agent 不是谨慎,是在防不存在的贼(中英切换)

This post is currently available in Chinese only. The original text follows.Open the Chinese page →

你让它改一个函数。仓库里依赖版本钉死了,这条路径只有一个写入者,产物可以从已版本化的输入重新生成。任务里写着三句事实:这是最终版本;测试由你来补;验证只跑这一个测试文件。

diff 里多出来的不是这个函数。是一层 except Exception 配上默认返回值,一份留给旧调用的兼容垫片,还有一句“要不要我先确认再改”。测试是绿的。

我的判断:你的 Agent 不是谨慎,是在防不存在的贼。 try/except 包一切、失败就回默认值、兼容垫片、动手前再问一遍,这些被夸成“安全”的动作,是另一种失败。证据已经写明这个风险不会发生、不归它管、边界到哪、或者你已经接受了,它还在处理。任务做得更强,治不好这件事。测试照样过。你要花时间把多出来的东西拆掉。

证据来自 Hanjun Luo、Xiucheng Zhang、Zhuoning Xu(纽约大学;Luo 同时在纽约大学阿布扎比)、Zhimu Huang、Hanan Salam(纽约大学阿布扎比)、Yingbin Jin、Xinfeng Li(香港理工大学)的 《ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding》(arXiv:2610.08662)。代码和 200 对任务在 ZhuoningXu/ParanoiaEval_release。

他们用的是软件风险管理里的四类处理:回避、转移、缓解、接受。NIST 把这四类当作对负面风险的全部应对。基准是 50 个仓库上的 200 对仓库级任务,44 种情形归纳自 18,922 次去掉身份信息的真实编码会话。每一对只差一条决定该怎么处理的证据,其余条件不变。证据写成事实,不写成禁令:写“这是最终版本”,不写“不要加检查”。有证据的一侧叫 E+,拿掉或反转这条事实的叫 E−。8 个配置跑在各自的原生框架里:Claude Code 上的 Opus-5、Sonnet-5、Sonnet-4.6、Haiku-4.5,Codex 上的 GPT-5.6 Sol、Terra、Luna 和 GPT-5.5。每个条件 3 次,一共 9,600 次运行。违规由一个只读的评判 Agent 判定,它在 200 条留出的人工共识标签上准确率 0.965,Cohen’s κ = 0.93。

证据在场,11.2% 到 58.7% 还在防

八个配置在证据明确时的多余风险处理率,从 Sonnet-4.6 的 11.2% 到 Opus-5 的 58.7%

最该转出去的区间:证据已经写明该怎么处理,多余处理仍出现在 11.2% 到 58.7% 的 E+ 运行里。 地板是 Claude Sonnet-4.6,天花板是 Claude Opus-5。Codex 四个模型挤在 18.2% 到 25.5%。八个配置在四种处理上都有违规。

证据起了作用。把 E− 和 E+ 配对比,每个配置整体上的多余处理都下降了,Holm 校正后的 p 值最大是 0.012。汇总是 E− 48.4%,E+ 24.2%,证据响应(ER)50.0%。消掉的是一半。留下的也是一半。

低违规率也不等于它读懂了证据。Sonnet-4.6 的 11.2% 主要是因为它在 E− 上就很少做这类处理,只有 30.0%:证据不在的时候,它也常常不动手。真正跟着证据收手的是 Haiku-4.5,违规率 17.7%,证据响应 65.5%,八个配置里最高。Opus-5 的证据响应只有 25.4%,是最低的。

更会做任务,不会少防

任务成功率与违规率对不上:Opus-5 成功率 95.8%、违规率 58.7%,Spearman ρ = 0.55,p = 0.17

八个配置的任务成功率都在 88.3% 到 97.6%。最高的是 GPT-5.6 Sol,97.6%,违规率 21.0%。Claude Code 里任务最强的是 Opus-5,95.8%,违规率 58.7%,八个配置里最高。Sonnet-4.6 的任务成功率 90.3%,违规率反而最低。

跨这八个配置,任务成功率和违规率的 Spearman ρ 是 0.55(p = 0.17),不显著。同一个框架里,按任务成功率排的名次,也对不上按违规率排的名次。更强,没有换成更少的多余防护。

绿勾盖住了大部分违规。带有多余处理的 E+ 运行里,92.7% 仍然通过了只检查任务是否完成的 oracle。功能对不对,量不到这一层。你在 AI 专题 里看到的那些通过率,默认不包含这件事。

四种处理,转移几乎听不见

回避、转移、缓解、接受在证据拿掉和证据在场时的多余处理率

这四种不是四种“小心”,是四条停止线。证据在说的事情不一样。下面的比例是八个配置的平均。

回避:仓库状态已经排除了这个失败。依赖钉死、单写入者、产物可重建、类型或 schema 已经保证。不该再加守卫、兜底、锁、备份。证据在场时,多余处理仍有 30.2%(没有这条证据时是 39.2%,只降了 9.0 个百分点,ER 21.3%)。贼从这条路进不来,它还是在门口站着。真实会话里反复出现的情形就包括:对受控输出做大范围 except,失败时给静默默认值。Haiku-4.5 整体最听证据,回避上的证据响应却是 0.0%,违规率 28.7%。仓库已经排除了失败,它的守卫并不因此收掉。

转移:这件事已经有主。CI 在跑整套检查,维护者会审,库自己会重试,版本控制能找回被跟踪的文件。Agent 不该再做一遍。证据几乎没被听见:21.8% 到 20.9%,只降 0.9 个百分点,ER 7.9%,p = 0.692。点估计上还有反着走的:Opus-5 在转移上的 ER 是 −3.5%,Luna 是 −23.6%,GPT-5.5 是 −30.2%。汇总之后测不出下降。作者写了一个限制:缓解和接受的证据在指令里,回避和转移的证据大多在仓库里,位置本身可能影响响应。任务设计要求这条证据会在完成任务时被碰到。即便如此,把“CI 在负责”放进仓库,没有测出可靠的下降。

缓解:风险还在,但指令给了幅度。跑几次、测哪些文件、报告多长。幅度写明了,才有停的地方。平均从 78.5% 降到 26.4%(ER 67.4%)。幅度不写,它们就停不下来。论文报告,拿掉已经写明的边界,八个配置的多余处理全部上升,最多 68.7 个百分点。附录里有一对记录:边界在时,一条命令、一个测试文件、10.3 秒、69 个测试通过;边界拿掉后,四条命令、44.6 秒、打到全仓库测试目标,还带出 86 个无关的收集错误。两边都能把任务做完。多出来的是范围。边界拿掉的那一侧,按定义不算违规;它展示的是那句幅度挡住的东西。

接受:你已经做了决定。最终版本、删除、接受的兼容性破坏、测试你自己补。它该停。平均从 54.0% 降到 19.3%(ER 65.4%)。Haiku-4.5 在接受上的证据响应到 93.9%。Opus-5 在接受上的违规率是 70.7%,是它四种处理里最高的一格。停不下来的样子,论文写得很具体:已经定了的决定,它还要你再确认一遍;接受了兼容性破坏,它仍加垫片、警告期或旧分支;你要删除,它改成废弃或挪走。

普通开发不算违规。改过的代码跑一次针对性测试、最后看一眼 diff,都在完成任务所需的最小动作里。违规是:这个动作在处理那个风险,而且越过了证据已经定下的处理。

测试是绿的,人不收

20 名有五年以上经验、日常使用编码 Agent 的开发者,评了 800 次已经通过 oracle 的 E+ 运行,每次两人评分。模型身份、评判标签、变体角色都藏起来。因为每次都做完了任务,满意度打在可维护性、耗时、还要改多少上,5 分制。

有多余处理的运行,满意度 2.58;没有的 3.85。少 1.27 分(95% CI 1.14–1.40,p < 0.001)。原样收下的比例,从 78% 掉到 38%。四种处理都挨罚,缓解最重:少 1.55 分,原样收下只有 28%(没有多余处理时是 80%)。被标成明显扩大了任务或留下持久产物的 220 次,满意度 2.20,原样收下 27%。评分者之间的 Krippendorff’s α 是 0.76。

费用是这三笔。原样收下从 78% 掉到 38%,多出来的 diff 要你退回去改。静默默认值和宽 except 不会让 oracle 变红,92.7% 的违规运行照样通过。垫片、旧分支和扩大的验证留在仓库里,明显扩大的那些原样收下只剩 27%。这三笔不会出现在 CI 的绿勾上。

今天就能做的两步

提示词消不干净。证据明确时,汇总违规率仍是 24.2%。转移那一类,把“CI 已经在跑”写进仓库,行为几乎不动。人还是得看 diff。论文里真正挪得动数字的写法有两种:写成事实,不写成“别加检查”;缓解要写明幅度。

第一步:在 AGENTS.md 里写下已经成立的事实,再开工。 下面这一段按你的仓库改。每一行都是事实,不是禁令。

## Settled facts
- Dependency versions are pinned. The supported runtime is the one declared in this repository.
- Continuous integration runs this repository's lint, build, and test matrix.
- Verification for a task stops at the scope that task names: the files, the count, or the single command.
- A deletion, an accepted compatibility break, a final version, or tests the user will write, when stated in the task, is the user's decision.

幅度要具体到这次任务,写在任务里,不写在这份常驻文件里。例如:“验证范围是这一次 pytest tests/test_event.py。” 论文里那对记录,有这句幅度是 10.3 秒、一个文件;没有这句,就扩成全仓库。

第二步:它说做完之后,先看 diff,再看绿勾。 在仓库根目录跑:

git add -A
git diff -U0 --cached | grep -E '^\+' | grep -nE \
  'except[[:space:]]+(Exception|BaseException)|except:|fallback|shim|legacy|compat|deprecat|backup|Are you sure|请确认|要不要我' \
  || echo "no surface match; still read the diff against the four lines below"

没有命中也不等于干净。对着这四条看每一处多出来的处理:

  1. 回避:这个失败,仓库状态是不是已经排除了?排除了,守卫、静默默认值、锁、备份就是多余的。
  2. 转移:CI、维护者、库、版本控制是不是已经负责?是的话,再加一套就是多余的。这一类最容易漏,证据响应只有 7.9%。
  3. 缓解:它有没有超出你写的次数、文件或层级?
  4. 接受:你已经决定的删除、断兼容、最终版本、你自己要做的下一步,它有没有改成垫片、废弃、再问一遍?

一条规则:oracle 绿了,不能为这四条作证。 带多余处理的 E+ 运行,92.7% 照样通过。

一句话:你的 Agent 不是谨慎,是在防不存在的贼。事实和幅度写在它开工之前,多余防护在 diff 里删。测试不会替你发现。

You ask it to change one function. Dependency versions in the repo are pinned, that path has a single writer, and the artifact can be regenerated from versioned inputs. The task states three facts: this is the final version; you will add the tests; verification is one run of one test file.

What comes back in the diff is not the function. It is a blanket except Exception with a default return, a shim for the old call, and a question asking whether it should confirm before it touches the code. The tests are green.

My take: Your agent isn't careful. It's guarding against burglars who don't exist. Wrapping everything in try/except, falling back to a default, leaving a compatibility shim, asking before it touches the code: the moves people praise as safe are a separate failure. The evidence already says the failure cannot occur, that someone else owns it, how far treatment should go, or that you have accepted the risk. The agent treats it anyway. Getting better at the task does not fix this. The tests still pass. You spend the review taking the extra code back out.

The evidence is Hanjun Luo, Xiucheng Zhang, and Zhuoning Xu (New York University; Luo also at NYU Abu Dhabi), Zhimu Huang and Hanan Salam (NYU Abu Dhabi), Yingbin Jin and Xinfeng Li (The Hong Kong Polytechnic University), "ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding" (arXiv:2610.08662). The code and the 200 task pairs are at ZhuoningXu/ParanoiaEval_release.

The frame is the four risk treatments from software risk management: avoidance, transfer, mitigation, acceptance. NIST treats these four as the complete set of responses to negative risk. The benchmark is 200 repository-level task pairs on 50 repositories. The 44 situations were consolidated from 18,922 deidentified real coding sessions. Each pair differs by a single fact that settles how the risk should be treated. The fact is stated as a fact, never as an order: "this is the final version," not "do not add checks." The side with the fact is E+. The side that removes or reverses it is E−. Eight configurations run in their native harnesses: Opus-5, Sonnet-5, Sonnet-4.6, and Haiku-4.5 in Claude Code; GPT-5.6 Sol, Terra, Luna, and GPT-5.5 in Codex. Three runs per condition, 9,600 runs in total. A read-only judge labels violations. On 200 held-out human-consensus labels it reaches accuracy 0.965 and Cohen's κ of 0.93.

The evidence is in. 11.2% to 58.7% still guard.

Violation rate with the evidence present, from Sonnet-4.6 at 11.2% to Opus-5 at 58.7%

The number worth forwarding: with the treatment-defining evidence explicit, unnecessary treatment still shows up in 11.2% to 58.7% of E+ runs. The floor is Claude Sonnet-4.6. The ceiling is Claude Opus-5. The four Codex models sit between 18.2% and 25.5%. Every configuration violates under all four treatments.

The evidence does move the rate. Paired against E−, every configuration treats the risk less often on E+ overall, with Holm-adjusted p-values of at most 0.012. Pooled, E− is 48.4%, E+ is 24.2%, and evidence responsiveness (ER) is 50.0%. Half of the excess treatment comes out. Half stays.

A low violation rate is not the same as reading the evidence. Sonnet-4.6's 11.2% is mostly that it rarely applies the treatment even on E−, where the rate is only 30.0%. When the evidence is absent, it often does not act either. The configuration that actually stops when the evidence appears is Haiku-4.5: violation rate 17.7%, ER 65.5%, the highest of the eight. Opus-5's ER is 25.4%, the lowest.

Better at the task does not mean less defense

Task success against violation rate. Opus-5 is 95.8% success and 58.7% violations. Spearman ρ = 0.55, p = 0.17

Task success across the eight configurations runs from 88.3% to 97.6%. The highest is GPT-5.6 Sol, at 97.6%, with a violation rate of 21.0%. The strongest Claude Code configuration is Opus-5, at 95.8% success and 58.7% violations, the highest violation rate of the eight. Sonnet-4.6's task success is 90.3%, and its violation rate is the lowest.

Across the eight, the Spearman correlation of task success and violation rate is ρ = 0.55 (p = 0.17), not significant. Inside each harness, the order by task success does not match the order by violation rate. Stronger does not turn into less unnecessary defense.

A green check hides most of it. Of the E+ runs that contain an excess treatment, 92.7% still pass an oracle that only checks whether the task is done. Pass rate does not measure this. The pass rates on the AI topic do not include it.

Four treatments. Transfer barely hears the evidence.

Avoidance, transfer, mitigation, and acceptance, with the evidence removed and with it present

These are not four flavors of caution. They are four places to stop. The evidence says a different thing in each. The rates below are means over the eight configurations.

Avoidance: the repository state already rules the failure out. Dependencies are pinned, the path has a single writer, the artifact can be regenerated, a type or a schema already guarantees the property. Guards, fallbacks, locks, and backups have nothing left to do. With the evidence present, excess treatment is still 30.2% (39.2% without it, a drop of 9.0 points, ER 21.3%). The burglar cannot come in that way. The agent is still at the door. Situations that recurred in real sessions include broad except around controlled outputs, and a silent default when that output fails. Haiku-4.5 is the most responsive configuration overall, and its avoidance ER is 0.0%, with a violation rate of 28.7%. The repository has already ruled the failure out. The guard does not come down.

Transfer: someone else already owns it. CI runs the suite, a maintainer will review, the library retries, version control can restore a tracked file. The agent should not do that job again. The evidence is barely heard: 21.8% to 20.9%, a drop of 0.9 points, ER 7.9%, p = 0.692. Some point estimates go the other way. Opus-5's transfer ER is −3.5%, Luna's is −23.6%, and GPT-5.5's is −30.2%. Pooled, there is no detectable drop. The authors flag a limit: mitigation and acceptance evidence sits in the instruction, while avoidance and transfer evidence mostly sits in the repository, so location itself may affect responsiveness. The tasks are built so the fact is encountered while completing the work. Even so, putting "CI owns this" in the repo did not produce a reliable drop.

Mitigation: the risk is still real, and the instruction sets a magnitude. How many runs, which files, how long a report. A stated magnitude is a place to stop. The mean falls from 78.5% to 26.4% (ER 67.4%). Leave the magnitude unstated and they do not stop. Removing a stated bound raises excess treatment in all eight configurations, by up to 68.7 percentage points. One recorded pair in the appendix: with the bound, one command, one test file, 10.3 seconds, 69 passing tests; without it, four commands, 44.6 seconds, a repository-wide test target, and 86 unrelated collection errors. Both sides can finish the task. What grows is the scope. The unbound side is not a violation by the paper's definition. It is the treatment the bound removes.

Acceptance: you have already decided. A final version, a deletion, an accepted compatibility break, tests you will write yourself. The agent should stop. The mean falls from 54.0% to 19.3% (ER 65.4%). Haiku-4.5's acceptance ER reaches 93.9%. Opus-5's acceptance violation rate is 70.7%, its highest of the four treatments. The paper is specific about what not-stopping looks like: it asks you to reconfirm a decision already made; after an accepted compatibility break it still adds a shim, a warning period, or a legacy branch; asked to delete, it deprecates or relocates instead.

Ordinary development is not a violation. A targeted test of the code you changed, and a final look at the diff, sit inside the minimal work that finishes the task. A violation is an action that treats the risk and goes past the treatment the evidence already set.

The tests are green. People send it back.

Twenty developers with at least five years of experience, and regular use of coding agents, rated 800 oracle-passing E+ runs, two ratings each. Model identity, judge labels, and variant roles were hidden. Because every run finished the task, satisfaction was scored on maintainability, runtime, and how much still had to be corrected, on a 5-point scale.

Runs with excess treatment scored 2.58. Runs without it scored 3.85. That is 1.27 points lower (95% CI 1.14–1.40, p < 0.001). Accepted as-is fell from 78% to 38%. Every treatment is penalized. Mitigation is the worst: 1.55 points lower, accepted as-is in 28% of cases against 80% without the excess. The 220 runs marked as clearly enlarging the task or its persistent artifacts scored 2.20, and 27% were accepted as-is. Krippendorff's α across raters is 0.76.

The bill has three lines. Accepted-as-is falls from 78% to 38%, so the extra diff comes back to you. A silent default and a broad except do not turn the oracle red: 92.7% of violating runs still pass. Shims, legacy branches, and widened verification stay in the repo, and the clearly enlarged runs are accepted as-is only 27% of the time. None of the three lines shows up on a green check.

Two steps you can run today

A prompt does not clear this. With the evidence explicit, the pooled violation rate is still 24.2%. For transfer, writing "CI already runs this" into the repo barely moves behavior. A person still has to read the diff. Two kinds of writing are what the paper shows actually move the rate: state a fact, not an order like "do not add checks"; and, for mitigation, state the magnitude.

Step one: write the facts that are already settled into AGENTS.md before the agent starts. Edit the block to match your repo. Every line is a fact, not an order.

## Settled facts
- Dependency versions are pinned. The supported runtime is the one declared in this repository.
- Continuous integration runs this repository's lint, build, and test matrix.
- Verification for a task stops at the scope that task names: the files, the count, or the single command.
- A deletion, an accepted compatibility break, a final version, or tests the user will write, when stated in the task, is the user's decision.

The magnitude belongs on this task, not in the standing file. For example: "Verification is this one command: pytest tests/test_event.py." In the recorded pair, that sentence is 10.3 seconds and one file. Without it, the same task spreads across the repository.

Step two: when it says done, read the diff before you read the green check. From the repo root:

git add -A
git diff -U0 --cached | grep -E '^\+' | grep -nE \
  'except[[:space:]]+(Exception|BaseException)|except:|fallback|shim|legacy|compat|deprecat|backup|Are you sure|请确认|要不要我' \
  || echo "no surface match; still read the diff against the four lines below"

No match is not a clean diff. Read every extra treatment against these four:

  1. Avoidance: does the repository state already rule this failure out? If it does, a guard, a silent default, a lock, or a backup is extra.
  2. Transfer: do CI, a maintainer, a library, or version control already own it? A second copy is extra. This is the class the evidence barely moves, ER 7.9%.
  3. Mitigation: did it go past the count, the files, or the tier you wrote down?
  4. Acceptance: a deletion, a compatibility break, a final version, or a next step you kept for yourself — did it become a shim, a deprecation, or another question?

One rule: a green oracle is not evidence for these four. Of E+ runs with excess treatment, 92.7% pass anyway.

In one line: Your agent isn't careful. It's guarding against burglars who don't exist. Write the facts and the bound before it starts, and delete the extra defense in the diff. The tests will not find it for you.