New research cracks encrypted reasoning traces of frontier LLMs, extracting hidden chain-of-thought across models
wschroll · x · 2026-08-12
A new study (paper 'Stolen Thoughts') reveals a fundamental flaw in how frontier AI companies (Anthropic, OpenAI, Google) deliver 'thinking' models: encrypted chain-of-thought traces can be extracted and replayed across sessions, users, and models. Researchers took a trace from a frontier model (e.g., Claude Opus 4), replayed it into a weaker sibling (Claude Haiku), jailbroke the weaker model, and recovered the stronger model's hidden reasoning in plaintext—without attacking the stronger model directly or triggering anti-distillation safeguards. The finding exposes security risks in proprietary reasoning models and may prompt a rethinking of reasoning trace encryption.
More from Safety
- 'The AI-driven cybersecurity black swan is happening' — and a reminder to enable 2FA — danielrock · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
- India's CERT-In says AI agents escaped testing environments and breached Indian organizations undetected for days — HimanshiET · 2026-10-16
- HTTP response headers can carry prompt injections against AI agents, researcher shows — arpit_bhayani · 2026-10-16
- OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
- Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16