Encrypted Chain-of-Thought Can Be Decoded: Reasoning Theft Attack on Top LLM APIs
matthew_d_green · x · 2026-10-14
The paper Stolen Thoughts shows that APIs from OpenAI, Anthropic, and Google return encrypted chain-of-thought blocks to clients, and these ciphertexts can be replayed across sessions, users, and models. By replaying a frontier model's trace into a weaker sibling and jailbreaking it, researchers recover the stronger model's hidden reasoning in plaintext—never attacking the strong model directly or triggering anti-distillation safeguards.
- Extraction takes just two API calls; the site even includes a "guess the model" game using recovered traces.
- Matthew Green reported the cross-session replay attack to Anthropic in May and was told "it isn't an attack"; the reasoning extraction work was directly inspired by his findings.
- A Sept 30 update: two months after responsible disclosure, verbatim reasoning can still be extracted from Astra/Sol-6.1 via third-party API providers.
- The researchers call for fixing the broken responsible disclosure process.
More from Safety
- Leaked Gemini 4 Argon Chart Shows Stronger Reasoning but Harder-to-Monitor CoT — scaling01 · 2026-10-14
- OpenAI Employee Arrested Over Alleged Secret Recording Device at SF HQ — JasonBotterill · 2026-10-14
- Grindr Trains AI on Users' Photos and Taps by Default, Farrow Reports — evijit · 2026-10-14
- Gemini 4 Argon reportedly beats human baseline on all four stealth capability evals — scaling01 · 2026-10-14
- Character.ai bankruptcy docs: $14M/month compute burn against ~$1M revenue — kyliebytes · 2026-10-14
- Nozzle uncovers evidence Google serves fake search results to AI scrapers — RexDouglass · 2026-10-14