Paper Decodes Hidden Reasoning Traces from Claude and GPT
Zealousideal_Sort74 · reddit · 2026-08-12
A recent paper demonstrates how to 'steal' 100% of reasoning tokens from proprietary LLM APIs. The author shares several key insights based on this finding:
- Benchmarking Inflation: Leaked reasoning traces show that Claude appears to 'memorize' benchmark questions like AIME, suggesting its public performance scores might be overstated.
- Overthinking is Normal: Quirks like strange wording or 'overthinking' frequently seen in open-source models are actually extremely common even in frontier proprietary models.
- Impact on Distillation: Some speculate this API gap was previously exploited to distill frontier models; closing it could slow down such distillation efforts.
The author concludes that open-source models aren't as far behind as they seem, and frontier models hold no 'secret sauce' beyond data, compute, and engineering.
More from Models
- OpenAI Voice Update Adds French, German, Hindi, Korean, Japanese Voices and Naturaler Preambles — shaunralston · 2026-10-16
- Why aren't labs training on hundreds of millions of videos for native audio/video? — imjustnewatai · 2026-10-16
- ChatGPT UI screenshot reveals Wallet coming soon plus Email, Calendar, Health and Finances integrations — jxnlco · 2026-10-16
- Why the next OpenAI and Anthropic models may still disappoint: no native audio or video input — imjustnewatai · 2026-10-16
- Thousands Hit by ChatGPT Standard Voice Mode Regression: Wrong Voice, Interruptions — shaunralston · 2026-10-16
- Gemini Argon 4 Quota Exhausted, Reset Countdown Shows Nearly 7 Days — ssh4net · 2026-10-16