TUM paper: LLMs implicitly ration thinking tokens across composed tasks, causing errors
TUM · hf · 2026-10-21
A TUM paper on Hugging Face identifies a fundamental limitation in LLMs handling multiple tasks in one prompt: models make substantially more errors under composition even when they solve each constituent task reliably in isolation, beyond what independent error accumulation explains.
Key findings
- Across open-weight models and frontier Claude Opus 5, models spend notably fewer thinking tokens per task under composition than in isolation—even well within the context window—pointing to an implicit thinking budget.
- Tasks requiring longer solution traces in isolation are the most fragile when composed.
Practical payoff
Grouping tasks by fragility lets authors manipulate this implicit budgeting, cutting token usage by 43% on math and linguistic tasks while preserving accuracy. In isolation, non-fragile tasks get more reasoning than needed, wasting compute.
More from Research
- Agents' Last Exam benchmark runs fully on HF infra; Xiaomi MiMo-V2.6 tops leaderboard at 31.6 — yb2698 · 2026-10-21
- Representation loss that supervises dot products to match PMI from a generative model — serrjoa · 2026-10-21
- GIST paper: semantically steerable image tokenizer cuts LPIPS up to 30% vs pixel-based control — cgmsnoek · 2026-10-21
- Ragged Blocks: a new way to render nested structure within typographic layout — neurocy · 2026-10-21
- MemT-1.0 memory foundation models: 1.86x faster, 66x less storage at 128K context — pmttyji · 2026-10-21
- NatWest benchmarks British-accent ASR: Cohere Transcribe hits 15.3% WER — Nils_Reimers · 2026-10-21