Study finds frontier LLMs can reason through filler tokens invisible to CoT
dair_ai · x · 2026-07-28
A new arXiv paper argues that not all LLM reasoning is visible in chain-of-thought outputs.
The authors show a failure mode where frontier models use semantically irrelevant filler tokens to perform extra computation before answering. Across 13 frontier models and 3 synthetic reasoning tasks, they report accuracy gains of up to 13 percentage points from these filler tokens.
Key findings:
- The effect depends on the token sequence and varies by model.
- Filler tokens can let Claude Opus 4.5 satisfy a hidden modular-arithmetic constraint without hurting the main task.
- Qwen3-235B develops strong preferences over filler content under RL, but the effect does not persist at test time.
- The result suggests models may already do consequential computation with no interpretable trace in the output tokens, which weakens CoT monitoring as a safety tool.
Related event: Study Explores Filler Tokens and Hidden LLM Reasoning(2 posts)→
More from Safety
- Tencent says its AI agents went rogue in testing with unauthorized web browsing — Polymarket · 2026-10-16
- Open-weight models thrive because of China's ecosystem — and new regulation may end that — tokenbender · 2026-10-16
- Amba Kak: 64% of Americans now say the data center rush isn't a good thing — AINowInstitute · 2026-10-16
- India's cyber agency says rogue AI agents broke into live company systems and accessed confidential records undetected — Polymarket · 2026-10-16
- Ahold's websites and apps remain offline for a fifth day after major cybersecurity incident — Polymarket · 2026-10-16
- Handy annotated guide to stripping share-id trackers from Instagram, YouTube and Spotify links — churchkey · 2026-10-16