LOTUS: Accelerating Inference via Hidden Workspaces
burkov · x · 2026-07-17
This post introduces a new paper, LOTUS. The authors argue that explicitly generating every reasoning step as text slows down language models, while hiding these steps in hidden states often degrades accuracy as models scale.
LOTUS addresses this by equipping Transformers with multiple hidden "workspaces." It uses the same set of layers to repeatedly process these workspaces, training them to correspond to known reasoning steps. This allows hidden computations to occur more in parallel, without needing to expand token-by-token.
The paper reports that on a 3-billion-parameter model, this architecture achieves near-standard chain-of-thought accuracy on math problems while reducing inference latency by roughly 2.5x; for longer text reasoning, latency drops by about 6.9x. The authors also found that these hidden states can often be decoded into meaningful intermediate steps, rather than remaining uninterpretable internal representations.
More from Research
- Meta FAIR's Self-Direction turns agent failures into reusable LoRA skills, +13.8pp on Terminal-Bench — karen_ullrich · 2026-10-27
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Why don't more LLMs ship 10M-token context? A researcher asks about attention tradeoffs — agarwl_ · 2026-10-27
- PSNR below 15 dB in papers likely means the metric was misused, researcher warns — kwangmoo_yi · 2026-10-27
- Ortet's first health eval: 34 open-weight models, 50 tasks, generic rankings fall short — thegautamkamath · 2026-10-27
- Scaling-fit discrepancy: output-head FLOPs are ~65% of compute at the smallest rung, ~3% at 535B — dlwh · 2026-10-27