ICL CIPHERS: Quantifying "Learning" in In-Context Learning via Substitution Ciphers
Zhouxiang Fang, Aayush Mishra, Muhan Gao, Anqi Liu, Daniel Khashabi
cs.CL
2025-04-28
JHU recodes ICL inputs with substitution ciphers; a bijection beats irreversible shuffle by 7.6 points on Amazon 20-shot, small but consistent on four datasets and six models.
In-context learning (ICL) is often described as two overlapping modes. Task retrieval (TR) means the model recognizes a task it already saw in pre-training from the prompt shape. Task learning (TL) means it actually picks up a new input-label mapping from the demonstrations. TL is the part people mean when they say few-shot ICL generalizes to new work. Almost every public benchmark already leaked into pre-training, so isolating TL is hard.
Prior probes mostly rewrite the labels, swapping "positive" for "1" or a random letter. The inputs stay readable English, so the model can still recover the task from syntax and template. This paper rewrites the inputs instead. It applies a substitution cipher to a subset of tokens, so the sentences look like word salad to a human, while a bijection keeps the underlying task well-defined.
A token-level map c replaces a fraction r of the vocabulary in both demonstrations and the test input. Two encodings share the same shuffled token set and differ only in invertibility.
Bijective cipher: each original token always maps to the same substitute. The map is reversible, so a model that infers the temporary dictionary can recover the task. Non-bijective cipher: every occurrence is replaced by a uniform draw from the shuffled set. The same word can become two different words, no inverse exists, and there is nothing consistent to learn.
The "learning" score is the accuracy gap between these two encodings, not the absolute score. Unciphered tokens can still trigger retrieval, so holding the shuffled set fixed isolates the effect of a learnable map.
A few controls keep the comparison honest. Tokens are bucketed into ten Wikipedia-frequency bands and shuffled only inside a band (Zipfian shuffling), so rare-token difficulty does not dominate. Demonstrations are priority-sampled to include the test example's substituted tokens. Special tokens and punctuation stay intact. Extreme shuffle rates are uninteresting: r near 0 barely changes the text, r near 1 makes the task ill-posed. Main runs sit at moderate r.
Tasks: SST-2 and Amazon sentiment, HellaSwag four-way completion, WinoGrande pronoun resolution. Models: Llama-3.1-8B, Qwen2.5-7B, OLMo-7B, Gemma-2-9B, plus Llama-3.1-8B-Instruct and Llama-3.1-70B. Metric is accuracy, averaged over three runs.
At 20-shot, bijective usually beats non-bijective. On Amazon (r=0.6), Llama-3.1-8B goes from 64.7% to 72.3% (+7.6). Qwen2.5 +5.3, OLMo +3.0, Gemma-2 +4.2. On SST-2 (r=0.5) the four-model gaps sit between 3.7 and 5.0 points. HellaSwag and WinoGrande use lower r and the gaps shrink; Gemma-2 is -0.5 on HellaSwag and OLMo is -0.3 on WinoGrande.
| Dataset | r | Llama-3.1-8B non-bijective | bijective (delta) |
| SST-2 | 0.5 | 58.3% | 63.1% (+4.8) |
| Amazon | 0.6 | 64.7% | 72.3% (+7.6) |
| HellaSwag | 0.3 | 29.7% | 31.9% (+2.2) |
| WinoGrande | 0.1 | 53.7% | 55.5% (+1.8) |
On Amazon the gap grows with more shots, from +4.7 at 5-shot to +10.1 at 50-shot (72.5% vs 82.6%). The largest bijective minus non-bijective gap sits around r in (0.4, 0.6). Instruction tuning lifts absolute accuracy; 70B lifts both encodings. The gap itself does not grow with alignment or scale. Restricting the shuffle to nouns leaves similar gaps on HellaSwag and WinoGrande.
Logit Lens on intermediate layers shows a weak but consistent pattern: under the bijection, deeper layers start ranking the substitute above the original token as more substitutions appear in context. The non-bijective condition has no such preference.
For people who evaluate ICL, this is a stricter contamination control than label randomization. A random bijection is vanishingly unlikely in pre-training, so absolute accuracy is not the evidence; the gap is. Nobody should ship this cipher as a product trick. It is a measuring stick. The honest reading is that on-the-fly learning exists and is small under a deliberately hostile encoding. Label-side probes test the output map; this tests the input map. They complement each other.
The authors flag three issues. Ciphered text leaves natural language, so part of the small gap is task difficulty. The largest model is Llama-3.1-70B; GPT-4 and Gemini are absent. Interpretability tried PatchScope and only Logit Lens produced a signal. Priority sampling also feeds demonstrations that contain the test item's substituted tokens, which is more helpful than a random draw. The model does not need to invert the full map; capturing an attribute such as sentiment is enough. The gap is closer to "did it pick up task-relevant substitutions" than to a complete cryptanalytic decode. Unciphered tokens can still leak the task family, so the metric is relative, not pure learning.