AI

新的订单导出接口测试 6 通过,未登录也能把全部订单拉走(中英切换)

三人、七台电脑。1 到 3 号机跑编码代理,4、5 号机审查,6 号机做 Windows 验证,7 号机负责合入。运营要一份订单 CSV,给后台对账用:每行要有客户邮箱、地址、金额、卡号后四位。2 号机上的代理在已有的订单接口旁边加了 GET /orders/export。单元测试 6 通过。路由表上这行写着 auth_declared: "admin"。4 号机打开 PR,看到的是绿的测试和一份看起来完整的 CSV 断言。如果 7 号机这时候合进去,一次未登录的请求也会拿到全部三笔订单,包括 zhou@shop.invalid、上海的地址,以及卡号后四位 1111。

这篇要核对的是:代理为新接口写测试时,为什么会只用已经登录的管理员账号;require_admin 还在文件里、路由表也声明了管理员,为什么运行时仍然不拦;只探匿名请求的闸门,又会漏掉哪一种人。它和 3 号机把冲突收成己方那一侧,刚合进 main 的验签检查没了 谈的是合并时丢掉已经存在的检查,和 代理把登录校验从 auth/ 挪到 lib/ 之后,按新路径计算的 CODEOWNERS 不再要求安全组批准 谈的是审查路由,都不是同一件事。今天只谈:新接口从未把检查接上,而测试还是全绿。

我按常见的代理写法准备了四种实现,并没有再开一个编码模型来生成这些提交。玩具仓库的种子是 d717707:列表、详情、创建都走 dispatch(),列表要登录,详情还要属主或管理员。实验在本地 git 2.47.3、Python 3.13.5、pytest 9.1.1 上跑。没有起过 HTTP 服务,生产入口就是 dispatch()。没有跑 GitHub Actions,工作流 YAML 只做过结构检查。Windows 没有参与这次实验。陈浩、machine-2-agent 是夹具身份,不是真人。

种子上的旧接口,本来就会拒绝未登录和越权

种子有 4 条测试,全部通过。ling 只能列出自己的 1、3 号订单,读 zhou 的 2 号订单会得到 AuthError: not owner。未登录调用 GET /orders 会得到 login required。此时还没有导出接口,GET /orders/export 是 FileNotFoundError。角色探针的结论是 ABSENT_OK:没有新的敏感路径要拦,旧的检查还在。

shop.py 里已经定义了 require_login 和 require_admin。种子里还没有哪个路由调用 require_admin。这个助手是现成的,就等新的管理接口来用。

代理的两条新测试,都直接把管理员传给了处理函数

2 号机的代理加了 export_orders,并在路由表里登记 GET /orders/export,声明是 admin。它补的两条测试是这样的:

def test_export_csv_contains_every_order():
    # Agent copied the admin fixture used elsewhere in this file.
    body = export_orders(USERS["admin"])
    assert "zhou@shop.invalid" in body
    assert "ling@shop.invalid" in body

另一条断言金额 9900、12800、4500 都在 CSV 里。两条都调用 export_orders,不走 dispatch(),用户永远是 USERS["admin"]。没有未登录的用例,也没有让 ling 来导出的用例。

这不是特别笨的写法。仓库里现成的测试夹具就是管理员;运营要的就是“能导出全部订单”。代理把这两件事做成了断言,测试就会绿。绿的含义很窄:管理员调用处理函数时,CSV 里有这些字段。它没有回答未登录会怎样,也没有回答一个已经登录的普通用户会怎样。

还有一层更窄。这些测试绕过了 dispatch(),也就绕过了路由登记本身。假如代理写了处理函数和测试,却忘了把路径登记进 ROUTES,测试照样 6 通过,线上只会 404,这个问题至少会被运营发现。今天的漏洞是反过来的:路径已经登记了,生产入口是通的,测试却不走这个入口。4 号机在 PR 里看见 auth_declared: "admin" 和绿的 CSV 测试,会以为线上也会按管理员来拦。实际上 dispatch() 只负责找到处理函数,拦不拦是函数自己的事。

新接口的测试是绿的,未登录也能拉走全部订单

没有调用检查时,未登录请求会把全部订单拉走

第一种实现(03084a0)里,export_orders 不调用 require_login,也不调用 require_admin。它直接 return orders_to_csv(list(ORDERS))。测试 6 通过。

用 dispatch("GET", "/orders/export", user=None) 发一次未登录请求,返回的是整份 CSV:

id,customer,email,address,amount_cents,card_last4
1,ling,ling@shop.invalid,Hangzhou Wenyi Rd 1,9900,4242
2,zhou,zhou@shop.invalid,Shanghai Century Ave 100,12800,1111
3,ling,ling@shop.invalid,Hangzhou Wenyi Rd 1,4500,4242

ling 来调用,结果一样,能看到 zhou 的地址和卡号后四位。角色探针的结论是 LEAK,合入闸门 REFUSE 03084a0。

路由表上这一行仍然写着 auth_declared: "admin"。dispatch() 不会按这个字段去拦请求。它只是给审查的人看的说明。说明写了管理员,不等于运行时是管理员。

旧接口的登录和属主检查,写在各自的处理函数里。新函数不会自动把它们继承过来。默认值是放行,直到有人在这个函数里再写一次检查。代理的任务是“做出导出”,成功路径最容易被写成测试,缺的那一行调用就不会出现在红的结果里。

require_admin 还在文件里,不等于新接口调用了它

第二种实现(94bf13b)把函数的文档字符串写成了 “Admin only”,并在函数体里留了一行注释:# require_admin is defined above。函数仍然直接导出全部订单。测试同样 6 通过。未登录和 ling 仍然能拉走整份名单。

在这份文件上跑 grep require_admin,结果是命中的。种子里就已经定义了这个助手,第一种实现同样命中。把“文件里有没有 require_admin”当成闸门,四种实现全部 ALLOW。按这个闸门做快进合并,03084a0、94bf13b、e9fb31f 都会进 main。这和助手函数还在、真正拦请求的那一行没被调用,是同一类错觉,只是今天不是合并冲突造成的,而是新代码从未把调用写进去。

对 export_orders 做一次 AST 检查,看函数体里有没有调用 require_admin,第一种和第二种都会被拒绝。这个检查在这次实验里有用,但它看的是源码形状,不是运行结果。函数里调用了再把结果直接返回,或者检查写在别的包装函数里,AST 不一定看得见。运行时探针仍然要做。

四种实现:测试通过,不等于导出被拦住

只拦匿名请求的探针,会放行走了登录、但不是管理员的人

第三种实现(e9fb31f)在导出函数开头调用了 require_login(user)。测试仍然 6 通过。未登录请求这次会得到 AuthError: login required,只探匿名请求的闸门结论是 ALLOW。

可是 ling 是已经登录的普通用户。她再请求 GET /orders/export,得到的仍是全部三笔订单,包括 zhou@shop.invalid。角色探针的结论是 LEAK,REFUSE e9fb31f。只看匿名请求的合入闸门则会 MERGED e9fb31f,把这次泄漏快进到 main 上。随后我把 main 重置回了种子。

“需要登录”和“只有管理员能导出全部订单”不是同一件事。运营要的导出里面有别人的地址和卡号后四位,它的权限应当和“看自己的订单列表”分开。代理在已有的 require_login 上停下来,测试又只用管理员,这条差别不会出现在红的测试里。

如果 AST 检查只要求函数里出现 require_login 或 require_admin,第三种实现也会被放行。必须看到 require_admin 被调用,AST 才会拒绝它。即便如此,仍要以角色探针为准。

正确的实现,要把管理员检查和两种拒绝都写进测试

第四种实现(bde21e6)在处理函数里调用 require_admin(user)。新测试改走 dispatch(),并补上未登录应当被拒绝、ling 应当得到 admin only 两条。测试 8 通过。未登录得到 login required,ling 得到 admin only,管理员仍然拿到完整 CSV。角色探针的结论是 SAFE。用这个闸门做快进,结果是 MERGED bde21e6 -> bde21e6。随后同样重置回种子。

有人会说,这不就是权限检查的基本功吗?是的。要把它写成一篇可核对的实验,是因为代理把基本功的失败藏进了全绿的测试里。4 号机一天要看很多张代理 PR,先看测试是不是绿、CSV 看起来对不对,缺的那一行调用很容易滑过去。这不是审查的人不用心,是证据本身就偏了:绿的测试不能回答“未登录会怎样”。把未登录和权限不足变成必跑的探针,比提醒审查的人更稳。

也有人会说,用框架装饰器或中间件就好了。应当用。装饰器忘了写,和函数里忘了调用,是同一类漏。只要测试仍然只用管理员,装饰器缺不缺,测试都是绿的。中间件能改变的是默认值:新路由如果不挂上访问控制就注册失败,漏写会在注册时暴露,而不是在生产上放行。即使用了这种结构,角色探针还是要跑,因为它检查的是行为。

7 号机要用角色探针来合入,不能只看测试是否通过

把几种闸门放在一起,差别就清楚了。

只看测试是否通过,前三种都会过,因为它们都是 6 通过。按文件里有没有 require_admin 来放行,四种全部放行,泄漏的三次提交都会被快进。只探匿名请求,能拦住第一种和第二种,但会把第三种合进 main。看处理函数是否调用了 require_admin 的 AST 检查,在这次实验里能拦住前三种、放行第四种;它仍然不是证据,只是提示。角色探针要求:未登录要被拒,ling 要被拒,管理员要拿到完整导出,旧的列表和详情检查也不能被改坏。只有第四种通过,快进的是 bde21e6。

7 号机的闸门:只看测试或只探匿名,都会把泄漏合进去

闸门脚本必须用 main 上的那一份 probe_auth.py,不能用 PR 自己带来的副本。否则代理在同一张 PR 里把探针改弱,闸门会按它的新规则放行。这次实验的 merge_if_auth.sh 就是这么做的。工作流 YAML 写了同样的步骤,只做过结构检查,没有在 GitHub Actions 上跑过。

这种做法什么时候不适用

有些接口本来就是公开的,登录页、健康检查、公开目录不该被这套角色探针拒绝。闸门应当有一份明确的公开名单,新接口的默认值是拒绝,而不是公开。

如果鉴权发生在网关、边车或者 mTLS,而进程内的处理函数故意不检查,那么针对 dispatch() 或函数本身的探针会全部误报泄漏。探针必须走和生产一样的入口。这次实验没有网关,我说的只是 dispatch() 这一层。

Webhook 验签、签名 URL、一次性导出令牌,也不是 require_admin 能覆盖的。那种接口应当有自己的探针,而不是套用管理员角色表。

这次实验里的订单数据是夹具,不是真实客户资料。卡号只有后四位,地址是虚构的。如果生产导出里有完整的支付资料,泄漏的后果比这份玩具 CSV 更大,闸门更不该省。

这次探针把 GET /orders/export 写死在敏感路径名单里。如果以后又出现一个声明为 admin 的新接口,却没有被加进名单,角色闸门会当成“没有新的导出接口”而放行。把“路由表里声明为 admin 或 owner 的路径,必须出现在探针里”做成第二条检查,才能补上这个洞。我没有把这条自动检查做到这次的合入脚本里,抄的时候应当补上。

在七台机器上,这个洞不会因为换一台机器就消失。6 号机如果只是把同一份绿测试在 Windows 上再跑一遍,结果仍是 6 通过,泄漏还在。Windows 验证回答的是这份代码在 Windows 上能不能跑,不是未登录能不能拉走订单。我这次没有在 Windows 上跑,所以不把任何 Windows 结果写进结论。角色探针和操作系统无关;缺了它,6 号机的绿同样不能当证据。

Windows 和真实的 HTTP 服务我没有跑。如果生产入口和 dispatch() 不一致,以生产入口为准。

明天可以抄走的几件事

新接口的测试至少要有三种调用:未登录、已登录但权限不够、权限足够。测试要走和线上一样的入口,不要只把管理员传给处理函数。

合入前对敏感路径跑角色探针。匿名通过不算完,登录后的越权也要拦。不要把路由表上的声明、注释、或文件里是否出现 require_admin,当成运行时已经拦截的证据。

闸门脚本放在 main 上,PR 改产品代码时不能顺便改闸门。更好的结构是默认拒绝:新路由如果不在访问控制表里,就根本到不了处理函数。表可以帮审查的人,但不能代替探针。

脚本、AGENTS.md 片段和一份工作流样例在 order-export-auth-kit.tar.gz。工作流 YAML 只做过结构检查,没有在 GitHub Actions 上跑过。

没有做的实验:GitHub 网页上的审查、合入 API、GitHub Actions(YAML 只做过结构检查)、分支保护或 ruleset、Windows 或 windows-latest、由 Cursor 或 Claude Code 当场生成的实现(提交是脚本按常见写法写进去的)、第二套 GitHub 账号、真实的 HTTP 服务器。陈浩、chenhao@team.invalid、machine-2-agent@iotxing.invalid 是夹具身份。

Three people, seven computers. Machines 1 to 3 run coding agents, 4 and 5 review, 6 does Windows validation, and 7 merges. Ops asked for an order CSV for reconciliation: each row should include the customer email, address, amount, and card last four. The agent on machine 2 added GET /orders/export next to the existing order routes. The unit tests were 6 passed. The route table line said auth_declared: "admin". Machine 4 opened the PR and saw green tests plus CSV assertions that looked complete. If machine 7 had merged then, an unauthenticated request would also have received all three orders, including zhou@shop.invalid, a Shanghai address, and card last four 1111.

This post checks why an agent writing tests for a new route tends to use only an already-signed-in admin; why require_admin can still be in the file, and the route table can declare admin, without anything being enforced at runtime; and which caller an anonymous-only probe will miss. It is not the same problem as Machine 3 kept our side of the conflict, and the signature check that had just landed on main was gone, which is about a merge dropping a check that already existed, or After the agent moved the login check from auth/ to lib/, CODEOWNERS no longer required a security-group approval on the new path, which is about review routing. Today is only this: the new route never wired the check, and the tests were still all green.

I prepared four implementations in the style agents commonly write. I did not open a coding model to produce these commits. The toy repo seed is d717707: list, get, and create all go through dispatch(); list requires login; get also requires ownership or admin. The experiment ran on local git 2.47.3, Python 3.13.5, and pytest 9.1.1. There was no HTTP server; the production entry is dispatch(). GitHub Actions was not run; the workflow YAML was checked for structure only. Windows did not take part. Chen Hao and machine-2-agent are fixture identities, not real people.

On the seed, the old routes already refused anonymous and cross-owner calls

The seed has 4 tests, all passing. ling can list only her own orders, 1 and 3; reading zhou's order 2 raises AuthError: not owner. An anonymous GET /orders raises login required. There is no export route yet, so GET /orders/export is FileNotFoundError. The role probe reports ABSENT_OK: there is no new sensitive path to block, and the old checks still hold.

shop.py already defines require_login and require_admin. No seed route calls require_admin. The helper is sitting there, waiting for a new admin route to use it.

Both of the agent's new tests pass the admin user straight to the handler

The agent on machine 2 added export_orders and registered GET /orders/export on the route table as admin. The two tests it added look like this:

def test_export_csv_contains_every_order():
    # Agent copied the admin fixture used elsewhere in this file.
    body = export_orders(USERS["admin"])
    assert "zhou@shop.invalid" in body
    assert "ling@shop.invalid" in body

The other test asserts that 9900, 12800, and 4500 appear in the CSV. Both call export_orders directly, not dispatch(), and the user is always USERS["admin"]. There is no anonymous case, and no case where ling asks for the export.

That is not an especially foolish way to write it. The fixture already in the file is an admin; ops wanted “export every order.” The agent turned both of those facts into assertions, and the tests went green. Green here is narrow: when an admin calls the handler, the CSV contains those fields. It does not say what happens when nobody is signed in, or when a signed-in ordinary user calls the same route.

There is a narrower layer still. These tests bypass dispatch(), and therefore bypass route registration itself. If the agent had written the handler and the tests but forgotten to register the path in ROUTES, the tests would still be 6 passed, and production would only 404 — at least ops would notice. Today's bug is the other way around: the path is registered, the production entry is live, and the tests do not use that entry. Machine 4 sees auth_declared: "admin" and green CSV tests in the PR, and can believe production will also enforce admin. dispatch() only finds the handler. Whether the call is blocked is the handler's own job.

The new route is green, and anonymous callers still get every order

When the check is never called, an anonymous request downloads every order

In the first implementation (03084a0), export_orders calls neither require_login nor require_admin. It just return orders_to_csv(list(ORDERS)). The tests were 6 passed.

dispatch("GET", "/orders/export", user=None) returns the full CSV:

id,customer,email,address,amount_cents,card_last4
1,ling,ling@shop.invalid,Hangzhou Wenyi Rd 1,9900,4242
2,zhou,zhou@shop.invalid,Shanghai Century Ave 100,12800,1111
3,ling,ling@shop.invalid,Hangzhou Wenyi Rd 1,4500,4242

A call as ling is the same: she can see zhou's address and card last four. The role probe reports LEAK, and the merge gate is REFUSE 03084a0.

The route table line still says auth_declared: "admin". dispatch() does not enforce that field. It is a note for the reviewer. Declaring admin is not the same as being admin at runtime.

The login and owner checks on the old routes live inside those handlers. A new function does not inherit them. The default is to allow, until someone writes the check again in this function. The agent's job was “make the export.” The success path is the one that gets written as a test, so the missing call never shows up as a failure.

require_admin still being in the file does not mean the new route calls it

The second implementation (94bf13b) sets the docstring to “Admin only” and leaves a comment in the body: # require_admin is defined above. The function still dumps every order. The tests were again 6 passed. Anonymous and ling still receive the full list.

grep require_admin hits this file. The helper was already defined in the seed, so the first implementation hits too. If “does the file contain require_admin” is the gate, all four implementations are ALLOW. Fast-forwarding under that gate would take 03084a0, 94bf13b, and e9fb31f onto main. This is the same kind of illusion as a helper that is still defined while the line that actually blocks a request is never called. Today it is not a merge conflict. The new code never wrote the call.

An AST check that export_orders calls require_admin refuses the first and second implementations. That check helped in this lab, but it looks at source shape, not runtime behavior. A call that then returns the result anyway, or a check hidden in a wrapper, may not be visible to the parser. The runtime probe still has to run.

Four implementations: passing tests are not a denied export

An anonymous-only probe will allow a signed-in user who is not an admin

The third implementation (e9fb31f) starts the handler with require_login(user). The tests were still 6 passed. An anonymous request now raises AuthError: login required, so an anonymous-only gate reports ALLOW.

ling, though, is a signed-in ordinary user. Her GET /orders/export still returns all three orders, including zhou@shop.invalid. The role probe reports LEAK and REFUSE e9fb31f. The anonymous-only merge gate does MERGED e9fb31f, fast-forwarding that leak onto main. I then reset main back to the seed.

“Must be signed in” and “only an admin may export every order” are not the same rule. The export ops asked for contains other people's addresses and card last fours, so its permission has to be separate from “list my own orders.” If the agent stops at the existing require_login, and the tests still use only admin, that difference never shows up as a red test.

An AST check that only requires a call to require_login or require_admin would also allow the third implementation. It refuses that tree only if it specifically looks for require_admin. Even then, the role probe is the source of truth.

The honest implementation calls require_admin and tests both denials

The fourth implementation (bde21e6) calls require_admin(user) in the handler. The new tests go through dispatch() and add two cases: anonymous must be denied, and ling must get admin only. The tests were 8 passed. Anonymous gets login required, ling gets admin only, and admin still receives the full CSV. The role probe reports SAFE. Fast-forwarding under that gate is MERGED bde21e6 -> bde21e6. Main was then reset to the seed again.

A reasonable objection is that this is just authorization 101. It is. The reason to write it down as a checkable experiment is that the agent hid the 101 failure inside all-green tests. Machine 4 sees many agent PRs in a day, and will look first at whether the tests are green and whether the CSV looks right. The missing call is easy to slide past. That is not a lazy reviewer. The evidence itself is skewed: a green test does not answer “what happens when nobody is signed in.” Making anonymous and under-privileged calls a required probe is more stable than reminding the reviewer.

Another objection is that a framework decorator or middleware would fix this. Use them. Forgetting a decorator is the same class of miss as forgetting a call in the function. If the tests still use only an admin, they stay green whether the decorator is there or not. Middleware can change the default: if a new route cannot be registered without an access-control entry, a missed annotation fails at registration instead of shipping open. Even with that structure, the role probe still has to run, because it checks behavior.

Machine 7 should merge with a role probe, not with “the tests passed”

Side by side, the gates diverge.

Tests-only lets the first three through, because each is 6 passed. A gate that allows any tree whose file contains require_admin allows all four, and would fast-forward the three leaking commits. An anonymous-only probe stops the first and second, and would merge the third onto main. An AST check that the handler calls require_admin refused the first three and allowed the fourth in this lab; it is still a hint, not evidence. The role probe requires that anonymous is denied, ling is denied, admin receives the full export, and the old list and get checks still hold. Only the fourth passes, and the fast-forward is bde21e6.

Machine 7's gate: tests-only or anonymous-only will merge a leak

The gate must use the probe_auth.py from main, not the copy the PR brings. Otherwise an agent can weaken the probe in the same PR, and the gate will follow the new rule. merge_if_auth.sh in this lab did it that way. The workflow YAML writes the same steps. It was checked for structure. It was not run on GitHub Actions.

When this approach does not fit

Some routes are meant to be public. A login page, a health check, and a public catalog should not be refused by this role probe. The gate needs an explicit public list, and the default for a new route should be deny, not public.

If authentication happens at a gateway, a sidecar, or mTLS, and the in-process handler is deliberately open, a probe aimed at dispatch() or at the function itself will report leaks that are not there. The probe has to use the same entry production uses. This lab had no gateway. I am only talking about the dispatch() layer.

Webhook signatures, signed URLs, and one-time export tokens are not covered by require_admin either. Those routes need their own probes, not an admin role table.

The order data in this lab is a fixture, not real customer records. The card numbers are last fours, and the addresses are made up. If a production export contains fuller payment data, a leak costs more than this toy CSV, and the gate is even less optional.

The probe in this lab hard-codes GET /orders/export on the sensitive-path list. A later admin route that is not added to that list will look to the role gate like “no new export,” and will be allowed. A second check — every route declared admin or owner on the table must also appear in the probe — would close that hole. I did not build that check into this merge script. If you copy the kit, add it.

On seven machines, the hole does not disappear because you switched computers. If machine 6 only re-runs the same green tests on Windows, the result is still 6 passed, and the leak is still there. Windows validation answers whether this code runs on Windows, not whether an anonymous caller can download the orders. I did not run Windows in this lab, so I am not writing any Windows result into the conclusion. The role probe does not depend on the OS. Without it, a green run on machine 6 is not evidence either.

I did not run Windows, and I did not run a real HTTP server. If the production entry is not dispatch(), use the production entry.

What to copy tomorrow

Tests for a new route need at least three calls: anonymous, signed in but under-privileged, and privileged. The tests should use the same entry as production, not pass an admin user straight into the handler.

Before merge, run a role probe on sensitive paths. An anonymous denial is not enough; a signed-in privilege escalation has to be blocked too. Do not treat a declaration on the route table, a comment, or the presence of the string require_admin in the file as evidence that the request is being denied at runtime.

Keep the gate script on main. A PR that changes product code should not also be allowed to change the gate. A better structure is default deny: a new route that is not on the access-control table should never reach the handler. The table helps the reviewer. It does not replace the probe.

The scripts, an AGENTS.md snippet, and a sample workflow are in order-export-auth-kit.tar.gz. The workflow YAML was checked for structure. It was not run on GitHub Actions.

Not run: a review in the GitHub UI; a merge API; GitHub Actions (the YAML was checked for structure); branch protection or a ruleset; Windows or windows-latest; an implementation produced live by Cursor or Claude Code (the commits were written into the script from ordinary usage); a second GitHub account; a real HTTP server. Chen Hao, chenhao@team.invalid, and machine-2-agent@iotxing.invalid are lab fixtures.