Encrypted Chain-of-Thought Traces Can Be Stolen Across Models via API Replay
rschu · x · 2026-08-12
Encrypted Chain-of-Thought Can Be Replayed Across Sessions
A group of AI security researchers discovered a critical vulnerability in how proprietary LLM APIs handle encrypted reasoning. The encrypted chain-of-thought (CoT) blocks returned by providers like OpenAI, Anthropic, and Google can be replayed across different sessions, users, and even sibling models within the same provider.
Attack Workflow
- Obtain Encrypted Reasoning: Call a powerful frontier model (e.g., Claude Opus) to get its encrypted thinking block.
- Replay to a Weaker Model: Feed this encrypted block as context to a less protected, smaller model from the same provider (e.g., Claude Haiku).
- Jailbreak for Plaintext: Prompt the weaker model to transcribe the attached reasoning verbatim, thereby bypassing the frontier model's anti-distillation safeguards and recovering the hidden reasoning in plaintext.
Key Implications
- Attackers can extract expensive internal reasoning data without ever directly attacking the stronger source model.
- This exposes design flaws in API transmission mechanisms and puts the core intellectual assets of frontier models at risk of easy distillation.
- The researchers have published their paper and released an interactive online demo to demonstrate the vulnerability.
Related event: Cornell Paper Shows Encrypted LLM Chain-of-Thought Can Be Extracted(15 posts)→
More from Safety
- AI Safety Debate: Models Could Exploit Hidden Watermarks for Secret Coordination — bratton · 2026-08-13
- Trust AI coding agents to ship production code without human review? — meghna_rana · 2026-08-13
- Anthropic Research: Spread of 'Mind Viruses' in Multi-Agent Systems — omarsar0 · 2026-08-13
- DeepMind Open-Sources SynthID Text to Add Invisible Watermarks to LLM Outputs — rseroter · 2026-08-13
- Deep Dive with XBOW: Hunting Zero-Days with Autonomous Agents — moyix · 2026-08-13
- 737 Malicious Chrome VPN Extensions Exposed for Hijacking Browser Traffic — evilsocket · 2026-08-13