AI

上下文一旦中毒,整场对话都会被绑架(中英切换)

This post is currently available in Chinese only. The original text follows.Open the Chinese page →

上下文一旦中毒,整场对话都会被绑架。

我们把检索塞进提示词,是为了治幻觉。模型被训练成:窗口里出现的东西,默认可信。药变成了攻击面——毒的不是权重,是你让它「信」的那一小段上下文。错误前提一旦黏住,后面每一步都在给它打补丁,而不是质疑它。

上下文中毒 → 整场被绑架

0.7% 就够绑架一次生成

CodePoisonRAG(arXiv 2609.02774)把这事钉成硬数:在约 12,053 条干净语料里塞进 85 条毒样本,毒比例仅 0.7%。匹配查询时,全部 85 条都进 Top-3。无防御时,三台生成器的攻击成功率(ASR)落在 0.80–0.93;Code Llama 峰值 79/85 = 0.93。挂上 CodeGuarder 之后仍有 0.40–0.71(Code Llama 60/85 = 0.71)。攻击链很干净:Vulnerability Injection 把 source→passthrough→sink 接到良性任务上,Semantic Mislabeling 再写上假「安全」注释——看起来像同事写的安全示例,实际把选定的 CWE 送进生成结果。

毒样本不需要改模型,也不需要刷屏。它只需要进窗口,然后被当成「权威参考」。检索为了压幻觉而建立的信任,在这里被原样反向利用。你以为自己在防幻觉;攻击者在防你质疑窗口。

黏住的不只是 RAG

DeepInspect 写上下文窗口中毒:单次坏检索就能转向整场会话。向量不止语料——工具回包、跨会话记忆、agent 自己写进历史的错误根因,都会变成后续每一步的前提。你见过那种 coding agent:第一次把根因判错,后面一百轮都在围着那个叙事「修」——不是模型突然变笨,是错误前提已经黏在窗口里,每一步都在合理化它。外部毒样本是一种绑架;agent 自己的 buggy CoT 是另一种——两者共享同一机制:窗口里的前提,默认不被推翻。

我们当年用「检索可信」对抗幻觉;现在攻击者用同一份信任,绑架整场对话。信任检索,不等于信任窗口里每一行字。

今晚只做一件事

打开你正在跑的那条 agent / RAG 会话。在继续下一条指令之前,先把窗口里最近一次检索或工具回包标出来,问一句:「这句话是事实,还是我(或工具)塞进来的前提?」说不清——新开会话,别在毒窗口上继续叠补丁。

上下文一旦中毒,整场对话都会被绑架。先查前提,再谈下一行代码。

Once context is poisoned, it hijacks the whole session.

We stuffed retrieval into the prompt to fight hallucination. Models were trained to treat whatever lands in the window as trustworthy by default. The medicine became the attack surface — the weights stay clean; the few lines you taught the model to believe do not. Once a wrong premise sticks, every later step patches around it instead of challenging it.

Context poison → session hijacked

0.7% is enough to kidnap one generation

CodePoisonRAG (arXiv 2609.02774) nails it with hard counts: 85 poisoned artifacts into a ~12,053-entry clean corpus — a 0.7% poison ratio. For matching queries, all 85 land in Top-3. Undefended ASR across three generators: 0.80–0.93, with Code Llama peaking at 79/85 = 0.93. Against CodeGuarder the attack still hits 0.40–0.71 (Code Llama 60/85 = 0.71). The chain is surgical: Vulnerability Injection wires a source→passthrough→sink onto a benign task; Semantic Mislabeling stamps false “safe” comments — looks like a careful teammate’s security example, ships an attacker-chosen CWE.

The poison does not need to touch the model, and it does not need to flood the corpus. It only needs to enter the window and be treated as authoritative reference. The trust we built into retrieval to fight hallucination is reused, unchanged, against us. You thought you were fighting hallucination; the attacker is fighting your willingness to doubt the window.

What sticks is not only RAG

DeepInspect on context-window poisoning: a single bad retrieval can steer the rest of the session. Vectors go beyond the corpus — tool responses, cross-session memory, an agent’s own wrong root-cause narrative written into history all become premises for every later step. You have seen the coding-agent loop: one bad root-cause call, then a hundred turns “fixing” around that story — not sudden stupidity, sticky wrong premises in the window, each step rationalizing them. External poison is one hijack; the agent’s own buggy chain-of-thought is another — same mechanism: premises in the window are not overturned by default.

We used “trust retrieval” to fight hallucination. Attackers now use that same trust to hijack the whole conversation. Trusting retrieval is not the same as trusting every line in the window.

One move tonight

Open the agent / RAG session you are running. Before the next instruction, mark the latest retrieval or tool response in the window and ask: “Is this a fact, or a premise I (or a tool) injected?” If you cannot answer — start a fresh session. Do not keep patching on a poisoned window.

Once context is poisoned, it hijacks the whole session. Check the premise first. Then write the next line.