Langford on FBT surprises: no nextlat needed as Recirculation paper cuts perplexity 23%
JohnCLangford · x · 2026-09-04
John Langford shares two surprises from developing the Full Bandwidth Transformer: nextlat proved unnecessary and training was easy. He points to the arXiv paper "Recirculation" (Mozer et al.), an inference-time architectural enhancement adding a specific recurrence so feedforward transformers can track belief states as dynamical systems, with essentially no added generation latency. The training-free adaptive variant, freezing original weights, cuts perplexity 23% and lifts GSM8k accuracy 21% on the Gemma3 family.
More from Research
- Astra produces Lean disproofs of Smale's and Köthe conjectures — basedjensen · 2026-09-05
- COLM paper: legible chain-of-thought steps aren't necessarily important — LauraRuis · 2026-09-05
- Allie Miller shares her AI research workflow: hypothesis-first with hundreds of agents — alliekmiller · 2026-09-05
- New Paper: LLM Forecasts Get More Incoherent as Logical Relations Between Events Increase — soumitrashukla9 · 2026-09-05
- Forecast Coherence Varies ~100x Across LLMs, and Higher Coherence Tracks Higher Accuracy — soumitrashukla9 · 2026-09-05
- Measuring LLM Forecast Incoherence via Arbitrage Profits: New Paper from Sarkar & Andrews — soumitrashukla9 · 2026-09-05