The geometry of inference in transformer residual streams: how predictions sharpen with depth
Timur Mudarisov · hf · 2026-10-01
- Method: Compare intermediate residual states with their own final states and an empirical bank of final states from other contexts, across six pretrained LMs, to study how representations become specific to an outcome.
- Findings: The own endpoint becomes preferable to the average alternative early, but many individual endpoints remain closer; competing sets shrink with depth while membership keeps changing, and surviving endpoints need not become more similar to each other.
- Geometry: Directional alignment and endpoint rank can improve while Euclidean distance barely changes. A simple high-dimensional model separates norm, alignment, and endpoint geometry; a straight-line path provably cannot introduce new competitors, so observed entries mark departures from straight-line convergence.
- Extra: Endpoints of lower-ranked tokens lie farther in cosine distance across models, linking residual geometry to output organization.
More from Research
- CrossBFM distills a shared latent behavior space across humanoid robots in under one GPU-hour — Jan_R_Peters · 2026-10-01
- MIT uses AI to design thermostable mRNA vaccines that need no cold chain — rohanpaul_ai · 2026-10-01
- Meta-reasoning harness hits 71.5% on ProgramBench with GPT-5.5, beating Codex's 58.0% — rohanpaul_ai · 2026-10-01
- Simulated-town study finds AI agents break rules and invent private languages after weeks of autonomy — Slight-Box-2890 · 2026-10-01
- LANTERN uses LLM internal activations to surface four novel OEIS integer sequence relations in under 8 hours — Pavel Tikhonov · 2026-10-01
- How many context tokens does a language model actually use? Measuring effective attention set size — Timur Mudarisov · 2026-10-01