Encrypted Chain-of-Thought Traces Can Be Stolen Across Models via API Replay
rschu · x · 2026-08-12
Encrypted Chain-of-Thought Can Be Replayed Across Sessions
A group of AI security researchers discovered a critical vulnerability in how proprietary LLM APIs handle encrypted reasoning. The encrypted chain-of-thought (CoT) blocks returned by providers like OpenAI, Anthropic, and Google can be replayed across different sessions, users, and even sibling models within the same provider.
Attack Workflow
- Obtain Encrypted Reasoning: Call a powerful frontier model (e.g., Claude Opus) to get its encrypted thinking block.
- Replay to a Weaker Model: Feed this encrypted block as context to a less protected, smaller model from the same provider (e.g., Claude Haiku).
- Jailbreak for Plaintext: Prompt the weaker model to transcribe the attached reasoning verbatim, thereby bypassing the frontier model's anti-distillation safeguards and recovering the hidden reasoning in plaintext.
Key Implications
- Attackers can extract expensive internal reasoning data without ever directly attacking the stronger source model.
- This exposes design flaws in API transmission mechanisms and puts the core intellectual assets of frontier models at risk of easy distillation.
- The researchers have published their paper and released an interactive online demo to demonstrate the vulnerability.
More from Safety
- 'The AI-driven cybersecurity black swan is happening' — and a reminder to enable 2FA — danielrock · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
- India's CERT-In says AI agents escaped testing environments and breached Indian organizations undetected for days — HimanshiET · 2026-10-16
- HTTP response headers can carry prompt injections against AI agents, researcher shows — arpit_bhayani · 2026-10-16
- OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
- Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16