LLMs can hold two thoughts at once, paper finds via embedding superposition
KyeGomezB · x · 2026-09-26
A paper highlighted by HuggingPapers shows that linear superposition is built into Transformers: averaging the embeddings of two texts makes an LLM predict both continuations at once in a single forward pass.
- Mechanism: the capability exists natively but fades during pretraining.
- Recovery: light fine-tuning can restore it.
- Result: the authors demonstrate decoding both content streams from one forward pass.
Related event: Study shows LLMs can naturally hold two thoughts at once(2 posts)→
More from Research
- Binary-reward RL hits 94% success on hard robotics task, recipe looks clear — kastnerkyle · 2026-11-11
- New paper digests OpenAI's Haldane-gap proof with tensor networks, formalizes it in Lean 4 — danintheory · 2026-11-11
- Stanford research featured by CNBC shows multi-agent AI teams fail to harness expertise — james_y_zou · 2026-11-11
- ARC Prize 2026 paper track closes with 708 submissions, results due December 4 — GregKamradt · 2026-11-11
- NeurIPS paper: [MASK] tokens betray jailbreaks in masked diffusion language models — kastnerkyle · 2026-11-11
- Interactive walkthrough of EAGLE-1: how one tiny layer drafts tokens to make LLMs 3x faster — hongyangzh · 2026-11-11