Study shows LLMs can naturally hold two thoughts at once

A new study proposes the superposition hypothesis: LLMs naturally output a linear superposition of next-token distributions when input embeddings from different texts are averaged, letting a single forward pass generate two coherent streams. The ability can be recovered after superposition fine-tuning.

2026-09-25 ~ 2026-09-26 · 2 related posts

• Transformers Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs — Pavel Tikhonov · 2026-09-25
• LLMs can hold two thoughts at once, paper finds via embedding superposition — KyeGomezB · 2026-09-26