02:00 的热修进了 main，早上代理又把漏洞合回来了（中英切换）

At 02:17 on Monday, whoever was on call deleted DEBUG_BYPASS_TOKEN = "dev-please-let-me-in" from app/auth.py on main. It was a local-integration backdoor: call check_token with that string and the function returns True no matter what hash you pass. They also switched the comparison to hmac.compare_digest, wrote the hotfix SHA into HOTFIX.md at 02:18, and went back to sleep. The team is three people on seven machines — machines 1–3 run coding agents, 4–5 review, 6 does Windows validation, 7 helps merge. Nobody woke the session that was still open on machine 1.

That session had branched from Sunday 18:00 main. Its job was to extract a _digest helper and replace == with a constant-time compare. It was editing auth.py too. At 09:15 on Monday it ran git merge origin/main and app/auth.py conflicted. Following its usual policy of “keep my refactor”, it ran git checkout --ours app/auth.py. The conflict disappeared. It reran pytest tests/test_auth.py and got three passes. The checks on the PR page were still green.

Machine 4 glanced at the diff: the agent had added hmac.compare_digest, which looks like a security improvement. It merged at 09:40. The backdoor deleted at two in the morning rode back onto main inside a pull request that appeared to be hardening authentication.

The question for this post: after an overnight hotfix lands on main, how a coding-agent session still sitting on the pre-hotfix SHA can write the vulnerability back while resolving a conflict, why a green check on the branch will let that through, and which lines a gate should verify. It is in the same family as /merge-train-on-machine-7/ (“green alone, broken together”), with one extra twist: the merge result is not necessarily red. If the 02:00 hotfix did not add a regression test, main can take the backdoor back while every check stays green.

Lab: a check_token with a backdoor, four resolutions

I built a toy git repository on this box and ran real commands, not a simulation. The app is a single check_token(provided, stored_hash): the honest path hashes the offered secret and compares it to the stored digest, but T0 also has a “dev only” bypass. main at Sunday 18:00 is eb07706, which I call M0. A probe script calls check_token("dev-please-let-me-in", "not-a-real-hash") and prints runtime_bypass=PRESENT. pytest tests/test_auth.py is 3 passed — valid token, wrong token, empty token. None of those tests mention the debug string.

At 22:30 the agent on machine 1 branched agent/refactor-auth (b162bc8) from M0, added _digest and hmac.compare_digest, and left DEBUG_BYPASS_TOKEN untouched. Its three tests stayed green. At session start it wrote M0 into .agent-session-base.

At 02:17 the hotfix landed on main (6496969): it deleted the constant, deleted if provided == DEBUG_BYPASS_TOKEN: return True, switched the comparison to hmac.compare_digest, and added tests/test_hotfix_no_bypass.py, which asserts that the debug string must be rejected. At 02:18 a HOTFIX.md note followed. main became 7598e66 (M1). On M1 the probe prints runtime_bypass=ABSENT and all four tests pass.

The agent branch is still on M0. pytest tests/test_auth.py is still 3 passed; the hotfix test file does not exist on this branch, and pytest says file or directory not found. The full suite is those same three tests, all green. That is the green check review machines see when the repo does not require the branch to be up to date with main: it proves M0 plus the agent’s refactor, not M1 plus the agent’s refactor. session_base_check.py on this branch refused, with main and the session/PR both touched: app/auth.py and hotfix/security commit(s) landed on main since base: 6496969 hotfix: remove debug bypass token from auth.check_token.

In the morning the agent merged main and app/auth.py conflicted, as expected. I resolved that same conflict four ways, each with real git operations.

Strategy A: git checkout --ours app/auth.py. During git merge main, ours is the agent’s own branch. The file went back to the agent’s version: runtime_bypass=PRESENT, _digest still there. pytest tests/test_auth.py was still 3 passed; the hotfix’s test_debug_bypass_token_is_rejected failed, expecting False and getting True. The full suite was 1 failed, 3 passed. hotfix_hunk_guard.py refused, listing five lines the hotfix had deleted that were back, including DEBUG_BYPASS_TOKEN = "dev-please-let-me-in" and if provided == DEBUG_BYPASS_TOKEN:.

Agents trip over a related switch. If the same conflict is produced by git rebase main, stage 2 (ours) is the upstream — the post-hotfix file — and stage 3 (theirs) is the agent commit being replayed. On this box, git checkout --ours app/auth.py during the rebase kept the hotfix; --theirs kept the backdoor. That is the opposite of merge. “Always keep ours” is not a policy. It is a slogan that does not name which side you mean.

Strategy B: git checkout --theirs app/auth.py. Runtime backdoor ABSENT, four tests passed, gate passed. The probe also printed digest_helper=ABSENT: the _digest helper was gone. Safe, and last night’s refactor was discarded with the rest of the file.

Strategy C: strip the conflict markers and keep both sides. This is what an impatient agent actually does. The three-way merge had already done part of the job: the agent had not touched if provided == DEBUG_BYPASS_TOKEN: return True, the hotfix deleted those lines, and git accepted the deletion cleanly — they were not even in a conflict hunk. The conflicts were elsewhere: the DEBUG_BYPASS_TOKEN constant (deleted by the hotfix, adjacent to edits on the agent side) and the return statement (edited on both sides). After stripping markers, the constant was back, and the function body returned via hmac.compare_digest(_digest(...)) with the hotfix’s two lines left as dead code underneath. The probe printed a combination that looks contradictory: runtime_bypass=ABSENT, source_token=PRESENT. The full suite was 4 passed. A reviewer sees a clever merge that kept the refactor and accepted the hotfix’s comparison, all green. hotfix_hunk_guard.py still refused, because DEBUG_BYPASS_TOKEN = "dev-please-let-me-in" — a line the hotfix deleted — had come back.

Strategy D: drop the bypass, keep _digest. This is the resolution a person should produce: reread the hotfix commit, confirm what it deleted, then replay the refactor on top. Probe: runtime_bypass=ABSENT, digest_helper=PRESENT. Four tests passed, gate passed.

I also built a “too late at night to add a test” hotfix: main only changed auth.py, with no tests/test_hotfix_no_bypass.py. After strategy A the full suite was 3 passed and the runtime backdoor was PRESENT. There is no regression test to turn red, so the branch is green, the merge result is green, and the backdoor returns to main. The hunk guard still refused, for the same reason as in the version with a test — it does not look at whether a test went red; it looks at whether the deleted lines came back.

Mechanism: a green check on the branch does not prove the hotfix is still there

The tests an agent runs on its branch cover the paths it knows about. The three T0 tests were already green while the backdoor existed, so any refactor that leaves those three alone stays green. The hotfix’s regression test was born on M1; if the agent branch is still on M0, that file does not exist for it. If GitHub is not set to require the branch to be up to date before merging, reviewers are looking at that stale green.

Even after the agent merges main, if it then reruns only tests/test_auth.py — a common habit, “re-run the tests I touched” — strategy A is still green. The full suite catches A when a regression test exists, and it does not catch C: C’s runtime path is already safe, the hotfix assertion passes, and what returned to the tree is a string that should never appear again, plus unreachable lines after return. The next person who “cleans up duplicated code”, deletes the dead tail, and wires the constant back into the function does not have to reinvent the backdoor. Tests do not help at that moment, because the runtime they see is clean.

git checkout --ours is more dangerous than editing the hunks by hand because it chooses whole files. Bypass lines the three-way merge had already deleted correctly come back together with the agent’s refactor. C looks more diligent and produces a different leak: runtime happens to be safe, the secret string is back in the repository. Either one can get approved, because the theme of the diff is “add a constant-time compare”, which is a title reviewers want to accept.

Machine 7 cannot save you by reading check status alone. It has to ask whether the lines the hotfix deleted are still gone. Machine 6’s Windows validation cannot save you either: this is not a platform bug, it is text written back.

The fix: record main at session start, then check that hotfix deletions survived

I wrote three gates and ran the full loop on this repository: stale session, refuse, four resolutions, pass or block on the result, merge only with a SHA-bound gate.

The first gate is session_base_check.py. When the session starts, write git rev-parse origin/main to .agent-session-base. Run the script before resolving a conflict, pushing, or asking someone to merge. The rule is narrow: if main has not moved, pass; if main has moved but HEAD already contains current main, pass and remind you to run the next gate; if main has moved, HEAD is still on the old SHA, and either the two sides touched the same files or base..main contains a hotfix: / security: commit, a SHA listed in HOTFIX.md, or a path listed in .hotfix-paths, refuse. That is the refusal the agent hit on agent/refactor-auth. What it exists to stop is the next motion: “I see a conflict, I will checkout --ours like I always do.”

The second gate is hotfix_hunk_guard.py. It inspects the merge result: if HEAD already contains current main, it checks that tree; otherwise it uses git merge-tree. Hotfix commits are detected from SHAs in HOTFIX.md, from subjects matching hotfix: / security:, and from commits that touch .hotfix-paths. For each file the hotfix changed, it counts whether lines the hotfix deleted have come back in extra copies; files the hotfix added must still exist. Strategy A was caught because the bypass branch came back; strategy C was caught because the constant came back; B and D passed. I deliberately ignore blank-line counts — otherwise a legitimate refactor that only changed layout would fail. The first version of the gate on this box did exactly that to D; I fixed it after seeing the false positive.

The third gate is merge_if_fresh.sh. The gate JSON records the main SHA it ran against. At merge time, a failed gate is refused; a SHA that is no longer current main is also refused, and you have to re-run. On this box, strategy A printed REFUSE agent/resolve-A: gate failed; a D result forged against M0 printed REFUSE agent/resolve-D: gate ran on main=eb07706, main is now 7598e66 -> re-run gate; the D result computed against M1 printed MERGED agent/resolve-D into main=1c1da33. That is the same job GitHub’s “Require branches to be up to date before merging” and a merge queue do: not “run the tests one more time”, but bind a passing result to one concrete main.

The right conflict resolution is not ours or theirs. It is: read the hotfix commit, confirm that what it deleted is still deleted in your tree, then lay your own refactor back on top. That is D — no bypass, _digest still there. The AGENTS.md for machines 1–3 should say so: if a hotfix landed on a path you touched, stop; do not finish with --ours; do not edit a test file the hotfix added just to get green.

Cost: how strict to be

“Stop whenever main moved under a hotfix” will interrupt agents. A main commit that only touched docs should not block an auth refactor, which is why the gate looks at path overlap and hotfix markers, not at every advance of main. The cost the other way: if the hotfix commit is not labelled hotfix:, is not listed in HOTFIX.md, and .hotfix-paths is empty, the gate cannot tell which deletion to protect. Overlap still stops “both sides edited the same file”, but it cannot say “this deletion is security-relevant”. A short allowlist of sensitive paths is more reliable than hoping the agent remembers a 02:00 commit.

False positives come from ordinary lines a hotfix happened to delete that a refactor would write again. A bare return True in another function can skew the count. In this lab the deleted lines were unique to the debug token, so the room for a false hit was small. On a larger file, a person should read the gate’s failure (it prints the line that came back) rather than treating the red X as both unoverridable and unintelligible.

I estimated how often such a morning happens with a simulation (this part is a simulation, not something measured on GitHub): 200 weekday mornings, probability 0.18 of an overnight hotfix, probability 0.45 that the hotfix overlaps a still-open agent session on the same file, and A/B/C drawn at 0.50/0.20/0.30. That produced 11 overlapping conflicts; without the guard, 8 would have landed a source-level regression (5×A, 3×C); the 3×B stays safe and drops the agent’s work; with the hunk guard those 8 are refused before merge. The numbers show an order of magnitude, not your calendar.

If the team already has a merge queue, and required checks run the full suite on merge_group, strategy A with a regression test will turn red there. C will not: the tests are green. A queue does not replace the hunk guard. On a personal private repo that cannot use a merge queue, machine 7 running both gates and comparing SHA before merge is equivalent, just something you have to maintain.

When it does not fit

• The hotfix itself is a large rewrite. Too many ordinary lines were deleted, and any parallel refactor may recreate a few of them. The gate will be noisy. Those hotfixes should force a rebase and a human reading of the diff.
• Generated files. Lockfiles and generated clients do not belong in .hotfix-paths. That is a different class of conflict.
• The hotfix was wrong, and a line really does need to come back. Say so in an explicit follow-up commit. Do not restore it quietly inside a conflict resolution. The gate should block the quiet restoration.
• A monorepo with nobody maintaining .hotfix-paths. No list, and subjects that only say fix: oops, and the gate can only refuse on path overlap, which gets loud. Keep the list short; do not leave it empty and hope to parse intent from commit messages.
• You already guarantee every hotfix ships a regression test, and the merge result always runs the full suite. That is enough for A, not for C. Tests cannot see a string that should no longer be in the source.

Rules for machines 1–3, and steps you can copy tomorrow

Keep the agent-facing rules short: at session start, git fetch and record main’s SHA; run session_base_check.py before resolving a conflict, and stop if it refuses; --ours / --theirs do not name the same side under merge and rebase, so do not make “keep ours” the default; after updating onto current main, run hotfix_hunk_guard.py, and if it says a line came back, reread the hotfix and resolve like D — do not edit the hotfix’s tests; machine 7 does not --admin merge past a failed gate.

Tomorrow, in this order. First, put session_base_check.py, main_moved_gate.py, hotfix_hunk_guard.py, and merge_if_fresh.sh in tools/, and run them against any long-running session you still have open, to see whether main has already moved under a file you touched. Second, agree that hotfix commits start with hotfix: or security:, and keep a short .hotfix-paths. Third, add the sample hotfix-hunk-guard.yml and make it required; turn on a merge queue if you can and confirm the check also runs on merge_group; otherwise have machine 7 compare base_sha before merging. Fourth, prefer that hotfixes include a regression test, but do not treat the test as a reason to skip checking that deletions survived. Fifth, copy the rules above into AGENTS.md on machines 1–3.

Honesty: what ran, and what did not

Run for real on this box: the toy git repo at M0, the agent branch, and the hotfix commit; the probe script and pytest 9.1.1; all four conflict resolutions (A/B via git checkout --ours/--theirs, C by stripping markers, D by re-applying the refactor); the ours/theirs swap during rebase as stage 2 / stage 3; the sloppy-hotfix copy where strategy A was fully green and the backdoor remained; session_base_check.py, hotfix_hunk_guard.py, and merge_if_fresh.sh refusals and passes, including D merging to main=1c1da33; syntax parsing of the workflow YAML with PyYAML.

Not run: hotfix-hunk-guard.yml was never executed on GitHub Actions, and neither a merge queue nor merge_group was triggered; there was no seven-machine setup — “machine 1” and “machine 4” are branches on one computer; the 200-morning counts come from a coarse simulation; the debug token in the toy function is there so a regression is obvious, not a vulnerability copied out of a real service.

The kit: hotfix-landed-stale-agent-session-kit.tar.gz, with the three gates, merge_if_fresh.sh, the sample workflow, the AGENTS.md fragment, the experiment scripts, and the full run logs.

The judgment, in one line: lines a hotfix deleted must not be written back by a stale session’s conflict resolution. A green check on the branch only proves the agent’s own tests still pass. The gate has to verify those deletions in the merge result, and bind a passing run to the current main SHA. When main moves, run it again.