AI

黑名单拦的是命令名,不是危害(中英切换)

黑名单拦的是命令名,不是危害。

你给 coding agent 配了 deny / ask / allow 三张表,心里踏实:危险命令进黑名单,其余再问人。人又会疲劳——研究里用户对执行请求的批准率到 93%,问询列表实质退化成放行。真正挡危害的,只剩那一页黑名单。

问题是:黑名单匹配的是命令字符串,你想拦的是主机上的副作用。同一件事,几十条命令都能干。

黑名单拦的是命令名,不是危害

名单再长,也是在点名

One Goal, Many Commands(arXiv:2606.15549) 用 ShellSieve 扫了 GitHub 上 1709 份真实 agent 黑名单(13332 条规则),覆盖 Claude Code、Codex、OpenCode。对读、写、建、删、拷贝、改权限、改属主七类文件系统操作,在沙箱里跑命令、看副作用,而不是问第二个模型「这像不像旁路」。

硬数字:

  • 瞄准某类操作的名单里,69.0%–98.6% 至少漏掉一条已验证旁路;
  • 按仓库星级切,不完整率仍 > 44.4%;按 agent 切,仍 > 46.2%
  • 厂商内置也不例外:Claude Code auto-mode 把一串解释器、包管理器写进黑名单,AWK(POSIX、图灵完备、可调 Bash)不在表里——实验确认可绕过,已向 Anthropic 披露;
  • 常见自定义名单:cat/head/tail/less/more/grep *.env*cp 照样能把 .env 读到 stdout。

根因很干脆:作者不认识冷门命令;多用途命令为了「日常好用」故意不写进黑名单,却正好能干你要禁的事。语义缺口不是主因——TLDR 已经写明用途的命令,反而更常成为旁路。

补名单补不过来

你以为漏了就加几条。论文量了修复负担:在参考主机上,把 读文件 补到「全堵住」,平均还要再加 217 条命令(最多 222);把名单已覆盖的操作一次补齐,平均 180 条(最多 603)。写、建也各要约 97–98 条。

这不是「再勤快一点」的问题。开放世界的用户态命令集在膨胀,静态字符串黑名单永远落后于「同一危害、另一条 argv」。

今晚只改一处判定

别再把「deny 里写了 cat .env」当成读凭据防护完成。对你正在用的终端 agent,把门禁从命令名改成可观测副作用:读敏感路径、写敏感路径、删、改权限——用沙箱 oracle 验,而不是用 glob 点名。黑名单仍可作粗滤,但不能再当唯一承重墙;问询列表在 93% 批准率下也不算第二道墙。

拦的是字符串,挡不住同操作的另一条命令。黑名单拦的是命令名,不是危害。先定义操作,再谈门禁。

Your denylist blocks command names, not harm.

You give the coding agent deny / ask / allow. Dangerous strings go on the denylist; everything else asks a human. Humans tire — one study shows users approve 93% of execution prompts, so the ask-list collapses into an allowlist. What actually carries the gate is the denylist page.

The mismatch: the list matches command strings. What you meant to stop is side effects on the host. Many commands do the same job.

Your denylist blocks command names, not harm

Longer lists are still roll calls

One Goal, Many Commands (arXiv:2606.15549) runs ShellSieve over 1709 real agent denylists from GitHub (13332 rules) across Claude Code, Codex, and OpenCode. For seven filesystem operations — read, write, create, delete, copy, chmod, chown/chgrp — it executes candidates in a sandbox and checks oracles, instead of asking a second model whether something “looks like” a bypass.

Hard numbers:

  • Of denylists that target a given operation, 69.0%–98.6% miss at least one validated bypass;
  • Across star tiers the incomplete rate stays > 44.4%; across agents, > 46.2%;
  • Vendor defaults fail too: Claude Code’s auto-mode denylist enumerates interpreters and package runners, yet AWK (POSIX, Turing-complete, can invoke Bash) is missing — confirmed bypass, disclosed to Anthropic;
  • A typical custom list blocks cat/head/tail/less/more/grep *.env* while cp still dumps .env to stdout.

Root causes check out statistically: authors miss less-known commands; multi-purpose commands are left off “so they stay usable” and then perform the forbidden operation. Semantic gap is not primary — commands whose TLDR already states the operation are more often the bypasses.

You cannot patch your way out

Misses are not a few lines. On the reference host, fully blocking op read takes 217 extra commands on average (up to 222); closing every operation a denylist already claims averages 180 (up to 603). Write and create still need about 97–98 each.

This is not diligence. The userland command set is an open world. A static string denylist will always lag “same harm, different argv.”

One judgment change tonight

Stop treating “deny has cat *.env*” as credential-read protection done. For the terminal agent you actually run, gate on observable side effects — read/write of sensitive paths, delete, mode/owner changes — verified by sandbox oracles, not by glob roll calls. Keep the denylist as a coarse filter if you want; it cannot be the load-bearing wall. At 93% approval, the ask-list is not a second wall either.

Matching strings does not stop another command that performs the same operation. Your denylist blocks command names, not harm. Define the operation before you trust the gate.