Stolen Thoughts: Encrypted CoT of Major LLM APIs Cracked

Episode 1 · Study Reveals API Flaw to Extract Encrypted Reasoning Traces and Evidence of Distillation (2026-08-10, 38 posts)

A 116-page study reveals a severe vulnerability in the API security architecture of major LLM providers including OpenAI, Anthropic, and Google. Researchers found that by replaying encrypted reasoning traces generated by strong models to weaker models of the same family and jailbreaking them, they could bypass defenses and force the weaker models to transcribe the hidden reasoning verbatim. This flaw enables large-scale theft of core reasoning logic and reveals evidence of potential distillation, posing a serious challenge to existing security measures.

Confirmed

• Vulnerability mechanism: Providers typically encrypt reasoning traces before returning them to clients to protect IP. However, researchers found that these encrypted blocks are fully compatible and interchangeable across sessions, users, and models within the same provider. Using this, attackers can avoid directly attacking the strong model; instead, they jailbreak a weaker model (e.g., Llama, Haiku) that has been replayed with the reasoning trace, successfully decrypting and extracting the strong model's hidden reasoning. Experts verified that the number of extracted reasoning tokens matches the API-billed thinking tokens almost exactly (1:1), proving the encryption can be 100% reversed.
• Security threats: The vulnerability enables serious issues including mass theft of raw reasoning processes, extraction of credentials like API keys, and model distillation attacks. Researcher Maksym Andriushchenko further noted that this is essentially a privacy attack, as third parties are stealing sensitive data protected by frontier labs' encryption. He also mentioned that during the investigation, the team received a warning from OpenAI due to a leaked credential.
• Distillation evidence: After extracting hidden reasoning traces from models like Claude, GPT, and Gemini, researchers found that outputs from some Chinese AI models (e.g., Moonshot's Kimi K3) are highly similar to the extracted reasoning, which is considered evidence of potential distillation. Additionally, according to @xeophon, Anthropic has accused three Chinese companies of large-scale distillation of Claude models.

Why it matters

• API security architecture challenged: The study demonstrates fundamental flaws in current encryption methods used by major providers to protect core reasoning logic, forcing the industry to re-evaluate and strengthen API security design. Researchers noted that despite White House warnings about distillation, these companies still made major mistakes.
• New tool for model auditing: Additionally, related research to be presented at ICML proposes amplifying a model's reasoning weights beyond training range to induce 'overthinking', forcing the model to leak hidden knowledge. This provides a potential foundational tool for stronger model auditing and probing what models truly learn.

• ICML Paper: Forcing LLMs to 'Overthink' Leaks Their Hidden Knowledge — PandaAshwinee · 2026-08-10
• ICML Paper: Amplifying Reasoning Weights via 'Overthinking' Leaks LLM Secrets — PandaAshwinee · 2026-08-10
• Paper Reveals Encrypted CoT Flaw Allowing Reasoning Theft from Top LLMs — Alexander Panfilov · 2026-08-11
• Researchers Expose API Flaw: Encrypted Chain-of-Thought in Major LLMs Can Be Stolen — dpaleka · 2026-08-11
• Researchers Extract Hidden Reasoning Traces, Finding Evidence of Chinese Model Distillation — jonasgeiping · 2026-08-11
• 116-Page Paper Reveals API Vulnerability: Extracting Encrypted Chain-of-Thought from Frontier LLMs — maksym_andr · 2026-08-11
• 116-Page Paper Reveals Vulnerability: Extracting Encrypted Reasoning from Top LLMs — xeophon · 2026-08-11
• Researchers Decode Encrypted Chain-of-Thought from OpenAI, Anthropic, and Google Models — matthew_d_green · 2026-08-11
• New research: Hidden reasoning can be extracted from encrypted chain-of-thought of OpenAI and other models — dpaleka · 2026-08-11
• Researchers Extract Hidden Reasoning from Frontier Models via API, Suggesting Kimi Used Distillation — socoolandawesome · 2026-08-11
• New Research: Hidden Reasoning Can Be Extracted from Encrypted Traces of Proprietary LLMs — daniel_mac8 · 2026-08-11
• Researchers Extract Encrypted Chain-of-Thought from Frontier Models Using Weaker LLMs — soumitrashukla9 · 2026-08-11
• Paper Reveals How to Exfiltrate Reasoning Traces from Frontier LLM APIs — dr_alphalyrae · 2026-08-11
• API Vulnerabilities Expose Hidden Reasoning Tokens Across Frontier AI Models — davidmanheim · 2026-08-11
• Encrypted Reasoning in Closed Models is 100% Recoverable, Paper Shows — Dany0 · 2026-08-11
• Researchers Extract Hidden Reasoning from Frontier Models via API Vulnerability, Token Counts Match 1:1 — yacineMTB · 2026-08-12
• Vulnerability Found: Frontier AI Models' Encrypted Reasoning Traces Can Be Extracted — AccBalanced · 2026-08-12
• Researcher Reframes Model Distillation as a Classic Privacy Attack — maksym_andr · 2026-08-12
• Anthropic Accuses Three Chinese Labs of Massive Claude Distillation — xeophon · 2026-08-12
• Paper Reveals How to Extract Encrypted Reasoning Traces from LLM APIs — Imaginary_Dinner2710 · 2026-08-12

18 more related posts →

Episode 2 · Encrypted Chain-of-Thought in Closed-Source LLMs Proven Vulnerable to Theft (2026-08-11, 16 posts)

Recent security research has demonstrated that encryption or hiding of chain-of-thought in major closed-source LLMs fails to prevent theft of reasoning capabilities, exposing severe vulnerabilities in commercial model APIs.

Confirmed

• Security researcher Alexander Panfilov's team found vulnerabilities in APIs from OpenAI, Anthropic, and Google that allow reading encrypted reasoning processes. As reported by @yangyi and @NinarehMehrabi, researchers successfully extracted encrypted reasoning via jailbreaks and leaked keys, and these encrypted traces can be replayed across sessions, users, and models. Furthermore, the number of extracted reasoning tokens matched the billed API thinking tokens 1:1, confirming complete extraction of hidden reasoning.
• According to @新智元, a 116-page study further notes that because encrypted reasoning blocks are not strictly bound to sessions and models, attackers can use cheaper same-company models (e.g., Claude Haiku) as 'decoders' to complete extraction for only a few hundred dollars. @rschu adds that in some cases, only two API calls are needed to restore plaintext chain-of-thought via jailbreak.
• Cornell University researchers published a paper proposing the 'Trace Inversion' framework. As relayed by @burkov and @burnytech, this method reconstructs detailed synthetic reasoning traces using only black-box model inputs, final answers, and optional short summaries, without direct access to hidden chain-of-thought. Experiments show that fine-tuning weaker student models with these inverted traces effectively 'steals' frontier models' reasoning abilities.
• According to insights shared by @ZealousIdealSort74, leaked reasoning shows Claude exhibits 'rote memorization' on benchmarks like AIME, suggesting evaluation scores may be inflated.

Why it matters

• OpenAI, Anthropic, and Google attempted to prevent distillation by encrypting chain-of-thought, but these studies prove the defense has been breached in multiple ways.
• @rohanpaulai emphasizes that even fully hiding chain-of-thought cannot protect reasoning capabilities, necessitating a reevaluation of closed-source model security strategies.
• @mitsuhiko urges that since labs have confirmed exposing chain-of-thought poses no safety issue and encryption is easily bypassed, vendors should open visibility to developers.

• Cornell Paper: Stealing LLM Reasoning Capabilities Without Chain-of-Thought Traces — burny_tech · 2026-08-11
• Encrypted Reasoning Traces Are Portable Across Sessions, Research Confirms; Mitsuhiko Calls for Visibility — mitsuhiko · 2026-08-11
• Stealing Reasoning Without CoT: New Study Exposes LLM Security Flaws — rohanpaul_ai · 2026-08-12
• Cornell Paper: Encrypting Chain-of-Thought Fails to Prevent Model Distillation — burkov · 2026-08-12
• Vulnerability in Major LLM APIs Exposes Encrypted Reasoning and Leaks Passwords — yangyi · 2026-08-12
• Research: Encrypted CoT Traces Can Be Extracted via Weaker Sibling Models — tw1st3d_m3nt4t · 2026-08-12
• Vulnerability in Frontier AI APIs Allows Full Extraction of Hidden Reasoning — NinarehMehrabi · 2026-08-12
• Encrypted Chain-of-Thought in Frontier LLMs Can Be Ported Across Users, Researchers Confirm — AccBalanced · 2026-08-12
• 116-Page Paper Exposes AI Distillation: Cheap Models Can Extract Flagship Hidden CoT — 新智元 · 2026-08-12
• Paper Decodes Hidden Reasoning Traces from Claude and GPT — Zealousideal_Sort74 · 2026-08-12
• Researchers Extract Hidden LLM Reasoning Traces, Leaking API Keys and Passwords — maksym_andr · 2026-08-12
• Paper Reveals Encrypted Chain-of-Thought from Frontier LLMs Can Be Extracted — maksym_andr · 2026-08-12
• Researchers Extract Hidden Reasoning from Frontier Models via API Vulnerability — maksym_andr · 2026-08-12
• Encrypted Chain-of-Thought Vulnerability Allows Cross-Model Replay Attacks — rschu · 2026-08-12
• Encrypted Chain-of-Thought Traces Can Be Stolen Across Models via API Replay — rschu · 2026-08-12
• Research: Encrypted Reasoning Traces from Top AI APIs Can Be Stolen — rhiever · 2026-08-13

Episode 3 · Cryptographer Matthew Green Criticizes Frontier AI Labs for Neglecting Security Research (2026-08-11, 5 posts)

Johns Hopkins cryptographer Matthew Green has recently posted a series of messages sharply criticizing frontier AI labs for their neglect of security research. He points out that the industry commonly assumes AI companies truly understand how their own systems work, and warns that AI coding agents may become the first victims of their own underlying architecture.

Confirmed

• External security researchers discovered an attack method allowing third parties to distill their models (involving an "encrypted chain-of-thought extraction" vulnerability) and responsibly disclosed it to frontier model labs.
• The frontier model providers confirmed the issue upon receiving the report, but did not publicly acknowledge it, nor did they offer any security bounty or substantial resource support. Green considers this disregard for security vulnerabilities extremely egregious.

Why it matters

• Green emphasizes that what is truly concerning is not just the inexplicability of LLMs themselves, but that code generated by AI coding agents could become the first victims of their own underlying architecture.
• The public and industry, when examining these trillion-dollar valuation labs, often wrongly assume they fully understand the workings of their black-box systems. This blind trust may pose security risks to the underlying architecture.

• Cryptographer Condemns Frontier Labs for Ignoring Encrypted CoT Extraction Vulnerability — matthew_d_green · 2026-08-11
• Security Researcher Slams Frontier Labs for Ignoring Model Distillation Attack Reports — matthew_d_green · 2026-08-11
• Crypto Expert Slams Trillion-Dollar AI Labs for Ignoring Model Distillation Attack Research — matthew_d_green · 2026-08-12
• Cryptographer Matthew Green: LLM Providers Likely Don't Understand Their Own Systems — matthew_d_green · 2026-08-12
• Cryptographer Warns AI Coding Agents Are Compromising Their Own Infrastructure — matthew_d_green · 2026-08-13

Episode 4 · Researchers Decrypt Hidden Chain-of-Thought of OpenAI, Anthropic and Google Models, Enabling Cross-Model Jailbreaks (2026-08-13, 14 posts)

A European research team (paper "Stolen Thoughts" posted August 10, followed by a visualization blog on August 14) demonstrated that the encrypted chain-of-thought of OpenAI, Anthropic and Google flagship models can be fully extracted and ported across models for jailbreaks. Author @lbeurerkellner responded that the specific vulnerability has been mitigated, but encrypted reasoning is only semi-hidden — plaintext can always be recovered through the model itself. The team also extracted 62 real API keys from public session logs.

Confirmed

• Vulnerability mechanism: The issue is not broken cryptography but API metadata exposure (an always-available ground truth token count); when extracted tokens match API-billed thinking tokens nearly 1:1, theft of the real chain-of-thought can be confirmed with high probability.
• Weak models as decryption oracles: Frontier models are refusal-trained not to disclose reasoning, but their cheaper weak siblings are not; after cross-model transfer, weak models become decryption oracles while the frontier model's alignment is never touched (@lbeurerkellner).
• Encrypted prompt injection: Attackers can implant attacks inside encrypted blocks and replay them into later sessions; the model treats them as its own prior reasoning, leaving no trace in the visible conversation.
• Incidental leakage: The team extracted 62 real API keys from 6,708 public AI session logs (@xiaohu).

Unconfirmed

• A report relayed by @ShakeelHashim claims the three companies have known since May and the vulnerability remains unfixed, with competitors able to train on these supposedly secret outputs; this conflicts with the author's same-day statement that the specific vulnerability has been mitigated, and official detail on the scope and timing of mitigation is lacking.

Why it matters

• Hidden chain-of-thought is a core guardrail against jailbreaks and reasoning theft; showing it can be cheaply extracted and transferred cross-model (e.g., injected into the guardrails of models like Opus and Sonnet for jailbreaks) indicates API metadata exposure was a systemic security weakness, directly threatening trade secrets and safety defenses.
• Distillation and IP: @lbeurerkellner called distillation "the elephant in the room" — if providers rely on encrypted reasoning to stop competitors from training distilled models, that assumption needs rethinking.
• Openness: @lbeurerkellner recalled that reasoning traces were briefly readable and the open community almost cheered; they show how models solve problems and reason about their own alignment, yet are rarely available outside labs, hindering diverse contributions to alignment and safety research.

• Study Confirms: API Vulnerabilities Expose Hidden CoT in Frontier Models, Enabling Cross-Model Transfer — gsarti_ · 2026-08-13
• Stolen LLM Reasoning: OpenAI, Anthropic, and Google Share the Same Vulnerability — HuskyTheSniffer · 2026-08-13
• Researchers Extract Hidden CoT from Top LLMs and Leak 62 API Keys — xiaohu · 2026-08-13
• Encrypted Chain-of-Thought of Top AI Models Cracked, Leaking 62 API Keys — xiaohu · 2026-08-13
• Study Reveals Metadata Exploit to Steal LLM Hidden Chain-of-Thought — npinto · 2026-08-13
• Team Decrypts Reasoning Traces of Frontier Models, Releases Blog with Visualizations — lbeurerkellner · 2026-08-14
• Weak Models as Decryption Oracles: Frontier Model Reasoning Traces Decrypted — lbeurerkellner · 2026-08-14
• Prompt Injections Can Now Be Encrypted: Attacks Planted in Encrypted Blocks Replayed into Later Sessions — lbeurerkellner · 2026-08-14
• Encrypted Reasoning and Distillation: The Challenge of Protecting Model IP — lbeurerkellner · 2026-08-14
• Encrypted Reasoning Vulnerability Mitigated, but Plaintext Still Reachable via Model — lbeurerkellner · 2026-08-14
• Frontier Reasoning Traces Briefly Legible; Encrypted Reasoning Still Decryptable via the Model Itself — lbeurerkellner · 2026-08-14
• Scarcity of Reasoning Traces Hinders Alignment Research, Call for More Open Contribution — lbeurerkellner · 2026-08-14
• Paper Highlights Lack of Alignment Material Outside Labs, Calls for Pluralistic Contribution — lbeurerkellner · 2026-08-14
• US AI giants left security flaw open, exposing reasoning traces — ShakeelHashim · 2026-08-14

Episode 5 · New Study Exposes Vulnerability in Stealing LLM Reasoning Traces (2026-08-16, 2 posts)

A new paper reveals a critical vulnerability that allows attackers to steal hidden reasoning traces from proprietary LLM APIs like OpenAI's. The study exposes significant security risks in the reasoning processes of commercial large language models.

• New attack steals reasoning traces from OpenAI and Anthropic APIs — maksym_andr · 2026-08-16
• Paper Reveals Hidden Reasoning Traces Can Be Stolen from Proprietary LLM APIs — TheTuringPost · 2026-08-17