HAPAX paper: abstractive ICL emerges even without induction-head copying
davidbau · x · 2026-10-26
EMNLP 2026 paper "In-Context Learning Beyond Copying" (arXiv:2511.05743) introduces HAPAX, a training regime that drops the loss contribution of tokens predictable by the induction circuit. Despite substantially reduced inductive copying and no characteristic loss phase changes, abstractive few-shot ICL still emerges, and models develop fewer, weaker induction heads — suggesting the developmental link between induction circuits and abstractive ICL is weaker than hypothesized. By Kerem Sahin, Sheridan Feucht, David Bau, Chris Wendler et al.
More from Research
- Two neurons flipped turn Qwen3.5-4B into an — repligate · 2026-10-26
- DirCast: one streaming video model directs and commentates live Dota 2 broadcasts — Zixian Gao · 2026-10-26
- UserArena benchmark uses 64,729 real shopping trajectories to test long-horizon agents — Jujia Zhao · 2026-10-26
- Echo teaches VLA robots to detect task success and failure without frame-level labels — Yufeng Liu · 2026-10-26
- ByteDance and universities open-source RSIHub, a composable agent-evolution framework — ByteDance · 2026-10-26
- Alibaba DAMO releases 16B unified embodied model RynnUnify, 51.5% on RoboCasa365 — Minghao Zhu · 2026-10-26