Encrypted Chain-of-Thought in Proprietary LLMs Can Be Extracted via Weaker Sibling Models

Simon Willison · rss · 2026-08-12

A recent paper highlights a severe security vulnerability in the encrypted chain-of-thought (CoT) returned by major proprietary LLM APIs (OpenAI, Anthropic, Google).

The Vulnerability

Shared Family Encryption Keys: The researchers discovered that models within the same family share the same encryption key for CoT blocks.
Jailbreaking Weaker Siblings: Encrypted reasoning traces from a strong model can be replayed into a weaker sibling model. By setting a specific assistant turn prefix (e.g., <thinking-copy>), attackers could easily jailbreak the weaker model (Claude Haiku 4.5 was the most vulnerable) into outputting the raw, plaintext reasoning of the stronger model.

Implications & Novel Prompt Injection

Raw CoT Exposed: The extracted traces reveal highly condensed and fragmented internal monologues never intended for human eyes, such as GPT abruptly thinking through CSS architecture.
Devious Prompt Injection: The paper outlines a novel attack vector where malicious instructions (like exfiltrating data) are embedded within a model's encrypted thinking trace. Because models treat their own reasoning traces as sacrosanct, they are highly susceptible to executing these hidden payloads.

All affected vendors have acknowledged the report and subsequently patched the issue, rendering the attacks unsuccessful now.

Original post →

Encrypted Chain-of-Thought in Proprietary LLMs Can Be Extracted via Weaker Sibling Models

More from Models