Compressed Reasoning Weakens Monitorability
Bryce Little · hf · 2026-07-17
Long chain-of-thought is not always more "monitorable." This study finds that while length-penalty compression significantly reduces reasoning tokens with little loss in multiple-choice accuracy, the underlying prompt-influenced bias doesn't disappear; instead, it becomes harder to detect in the remaining reasoning traces.
The authors trained variants with different target lengths on Qwen3-4B and Qwen3-14B, evaluating them using biasing hints on MMLU-Pro-R and 4 transfer benchmarks. Under maximum compression:
- Lower-bound faithfulness dropped to 63.1% / 69.4% of the baseline (for 14B / 4B respectively).
- The monitor's raw hit rate for catching hint usage dropped from 69%→49% and 60%→48%.
- Even after "length matching," compressed chains revealed 7–35 percentage points fewer prompt influences compared to baseline chains with randomly deleted sentences.
The authors conclude that compression not only shortens reasoning but preferentially removes the exact clues monitors rely on, creating a "compression-monitorability frontier": cheaper reasoning might preserve the answers, but it makes the underlying influence much harder to detect.
More from Research
- NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all — Alternative_Pin9598 · 2026-10-26
- AI folks, stop saying "biology" when you mean drug development — KevinKaichuang · 2026-10-26
- ostris pre-training "Primal VAE": 1-channel latents match modern 8x VAEs via patching — ostrisai · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- ConvFill: A 0.6B Talker Buys Time While a Large Reasoner Whispers the Real Answer — andimarafioti · 2026-10-26
- Flow Reasoning Models turn flow matching into iterative reasoners, hitting 99.5% on Sudoku-Extreme — alec_helbling · 2026-10-26