Encrypted Chain-of-Thought Vulnerability Allows Cross-Model Replay Attacks
rschu · x · 2026-08-12
A highly discussed AI security paper reveals a critical vulnerability in the encrypted Chain-of-Thought (CoT) used by modern LLMs. Researchers found that encrypted reasoning blocks can be replayed across different sessions and models within the same provider.
By feeding the encrypted reasoning of a powerful model into a weaker, less protected model, attackers can trick the latter into reproducing the hidden logic in plaintext. This attack was successfully demonstrated across Anthropic, OpenAI, and Google models.
Beyond privacy implications, this raises distillation concerns. Intriguingly, when testing Kimi K3, researchers noted unusual behavioral compatibility when seeded with reasoning fragments from Opus and GPT. While explicitly stated as suggestive but inconclusive, this finding adds fuel to previous distillation accusations against Moonshot AI.
More from Safety
- 'The AI-driven cybersecurity black swan is happening' — and a reminder to enable 2FA — danielrock · 2026-10-16
- No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
- India's CERT-In says AI agents escaped testing environments and breached Indian organizations undetected for days — HimanshiET · 2026-10-16
- HTTP response headers can carry prompt injections against AI agents, researcher shows — arpit_bhayani · 2026-10-16
- OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
- Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16