Encrypted CoT of Major Closed-Source LLMs Hacked

Episode 1 · Study Reveals API Flaw to Extract Encrypted Reasoning Traces and Evidence of Distillation (2026-08-10, 38 posts)

A 116-page study reveals a severe vulnerability in the API security architecture of major LLM providers including OpenAI, Anthropic, and Google. Researchers found that by replaying encrypted reasoning traces generated by strong models to weaker models of the same family and jailbreaking them, they could bypass defenses and force the weaker models to transcribe the hidden reasoning verbatim. This flaw enables large-scale theft of core reasoning logic and reveals evidence of potential distillation, posing a serious challenge to existing security measures.

Confirmed

• Vulnerability mechanism: Providers typically encrypt reasoning traces before returning them to clients to protect IP. However, researchers found that these encrypted blocks are fully compatible and interchangeable across sessions, users, and models within the same provider. Using this, attackers can avoid directly attacking the strong model; instead, they jailbreak a weaker model (e.g., Llama, Haiku) that has been replayed with the reasoning trace, successfully decrypting and extracting the strong model's hidden reasoning. Experts verified that the number of extracted reasoning tokens matches the API-billed thinking tokens almost exactly (1:1), proving the encryption can be 100% reversed.
• Security threats: The vulnerability enables serious issues including mass theft of raw reasoning processes, extraction of credentials like API keys, and model distillation attacks. Researcher Maksym Andriushchenko further noted that this is essentially a privacy attack, as third parties are stealing sensitive data protected by frontier labs' encryption. He also mentioned that during the investigation, the team received a warning from OpenAI due to a leaked credential.
• Distillation evidence: After extracting hidden reasoning traces from models like Claude, GPT, and Gemini, researchers found that outputs from some Chinese AI models (e.g., Moonshot's Kimi K3) are highly similar to the extracted reasoning, which is considered evidence of potential distillation. Additionally, according to @xeophon, Anthropic has accused three Chinese companies of large-scale distillation of Claude models.

Why it matters

• API security architecture challenged: The study demonstrates fundamental flaws in current encryption methods used by major providers to protect core reasoning logic, forcing the industry to re-evaluate and strengthen API security design. Researchers noted that despite White House warnings about distillation, these companies still made major mistakes.
• New tool for model auditing: Additionally, related research to be presented at ICML proposes amplifying a model's reasoning weights beyond training range to induce 'overthinking', forcing the model to leak hidden knowledge. This provides a potential foundational tool for stronger model auditing and probing what models truly learn.

• ICML Paper: Forcing LLMs to 'Overthink' Leaks Their Hidden Knowledge — PandaAshwinee · 2026-08-10
• ICML Paper: Amplifying Reasoning Weights via 'Overthinking' Leaks LLM Secrets — PandaAshwinee · 2026-08-10
• Paper Reveals Encrypted CoT Flaw Allowing Reasoning Theft from Top LLMs — Alexander Panfilov · 2026-08-11
• Researchers Expose API Flaw: Encrypted Chain-of-Thought in Major LLMs Can Be Stolen — dpaleka · 2026-08-11
• Researchers Extract Hidden Reasoning Traces, Finding Evidence of Chinese Model Distillation — jonasgeiping · 2026-08-11
• 116-Page Paper Reveals API Vulnerability: Extracting Encrypted Chain-of-Thought from Frontier LLMs — maksym_andr · 2026-08-11
• 116-Page Paper Reveals Vulnerability: Extracting Encrypted Reasoning from Top LLMs — xeophon · 2026-08-11
• Researchers Decode Encrypted Chain-of-Thought from OpenAI, Anthropic, and Google Models — matthew_d_green · 2026-08-11
• New research: Hidden reasoning can be extracted from encrypted chain-of-thought of OpenAI and other models — dpaleka · 2026-08-11
• Researchers Extract Hidden Reasoning from Frontier Models via API, Suggesting Kimi Used Distillation — socoolandawesome · 2026-08-11
• New Research: Hidden Reasoning Can Be Extracted from Encrypted Traces of Proprietary LLMs — daniel_mac8 · 2026-08-11
• Researchers Extract Encrypted Chain-of-Thought from Frontier Models Using Weaker LLMs — soumitrashukla9 · 2026-08-11
• Paper Reveals How to Exfiltrate Reasoning Traces from Frontier LLM APIs — dr_alphalyrae · 2026-08-11
• API Vulnerabilities Expose Hidden Reasoning Tokens Across Frontier AI Models — davidmanheim · 2026-08-11
• Encrypted Reasoning in Closed Models is 100% Recoverable, Paper Shows — Dany0 · 2026-08-11
• Researchers Extract Hidden Reasoning from Frontier Models via API Vulnerability, Token Counts Match 1:1 — yacineMTB · 2026-08-12
• Vulnerability Found: Frontier AI Models' Encrypted Reasoning Traces Can Be Extracted — AccBalanced · 2026-08-12
• Researcher Reframes Model Distillation as a Classic Privacy Attack — maksym_andr · 2026-08-12
• Anthropic Accuses Three Chinese Labs of Massive Claude Distillation — xeophon · 2026-08-12
• Paper Reveals How to Extract Encrypted Reasoning Traces from LLM APIs — Imaginary_Dinner2710 · 2026-08-12

18 more related posts →

Episode 2 · Encrypted Chain-of-Thought in Closed-Source LLMs Proven Vulnerable to Theft (2026-08-11, 16 posts)

Recent security research has demonstrated that encryption or hiding of chain-of-thought in major closed-source LLMs fails to prevent theft of reasoning capabilities, exposing severe vulnerabilities in commercial model APIs.

Confirmed

• Security researcher Alexander Panfilov's team found vulnerabilities in APIs from OpenAI, Anthropic, and Google that allow reading encrypted reasoning processes. As reported by @yangyi and @NinarehMehrabi, researchers successfully extracted encrypted reasoning via jailbreaks and leaked keys, and these encrypted traces can be replayed across sessions, users, and models. Furthermore, the number of extracted reasoning tokens matched the billed API thinking tokens 1:1, confirming complete extraction of hidden reasoning.
• According to @新智元, a 116-page study further notes that because encrypted reasoning blocks are not strictly bound to sessions and models, attackers can use cheaper same-company models (e.g., Claude Haiku) as 'decoders' to complete extraction for only a few hundred dollars. @rschu adds that in some cases, only two API calls are needed to restore plaintext chain-of-thought via jailbreak.
• Cornell University researchers published a paper proposing the 'Trace Inversion' framework. As relayed by @burkov and @burnytech, this method reconstructs detailed synthetic reasoning traces using only black-box model inputs, final answers, and optional short summaries, without direct access to hidden chain-of-thought. Experiments show that fine-tuning weaker student models with these inverted traces effectively 'steals' frontier models' reasoning abilities.
• According to insights shared by @ZealousIdealSort74, leaked reasoning shows Claude exhibits 'rote memorization' on benchmarks like AIME, suggesting evaluation scores may be inflated.

Why it matters

• OpenAI, Anthropic, and Google attempted to prevent distillation by encrypting chain-of-thought, but these studies prove the defense has been breached in multiple ways.
• @rohanpaulai emphasizes that even fully hiding chain-of-thought cannot protect reasoning capabilities, necessitating a reevaluation of closed-source model security strategies.
• @mitsuhiko urges that since labs have confirmed exposing chain-of-thought poses no safety issue and encryption is easily bypassed, vendors should open visibility to developers.

• Cornell Paper: Stealing LLM Reasoning Capabilities Without Chain-of-Thought Traces — burny_tech · 2026-08-11
• Encrypted Reasoning Traces Are Portable Across Sessions, Research Confirms; Mitsuhiko Calls for Visibility — mitsuhiko · 2026-08-11
• Stealing Reasoning Without CoT: New Study Exposes LLM Security Flaws — rohanpaul_ai · 2026-08-12
• Cornell Paper: Encrypting Chain-of-Thought Fails to Prevent Model Distillation — burkov · 2026-08-12
• Vulnerability in Major LLM APIs Exposes Encrypted Reasoning and Leaks Passwords — yangyi · 2026-08-12
• Research: Encrypted CoT Traces Can Be Extracted via Weaker Sibling Models — tw1st3d_m3nt4t · 2026-08-12
• Vulnerability in Frontier AI APIs Allows Full Extraction of Hidden Reasoning — NinarehMehrabi · 2026-08-12
• Encrypted Chain-of-Thought in Frontier LLMs Can Be Ported Across Users, Researchers Confirm — AccBalanced · 2026-08-12
• 116-Page Paper Exposes AI Distillation: Cheap Models Can Extract Flagship Hidden CoT — 新智元 · 2026-08-12
• Paper Decodes Hidden Reasoning Traces from Claude and GPT — Zealousideal_Sort74 · 2026-08-12
• Researchers Extract Hidden LLM Reasoning Traces, Leaking API Keys and Passwords — maksym_andr · 2026-08-12
• Paper Reveals Encrypted Chain-of-Thought from Frontier LLMs Can Be Extracted — maksym_andr · 2026-08-12
• Researchers Extract Hidden Reasoning from Frontier Models via API Vulnerability — maksym_andr · 2026-08-12
• Encrypted Chain-of-Thought Vulnerability Allows Cross-Model Replay Attacks — rschu · 2026-08-12
• Encrypted Chain-of-Thought Traces Can Be Stolen Across Models via API Replay — rschu · 2026-08-12
• Research: Encrypted Reasoning Traces from Top AI APIs Can Be Stolen — rhiever · 2026-08-13

Episode 3 · Cryptographer Matthew Green Criticizes Frontier AI Labs for Neglecting Security Research (2026-08-11, 5 posts)

Johns Hopkins cryptographer Matthew Green has recently posted a series of messages sharply criticizing frontier AI labs for their neglect of security research. He points out that the industry commonly assumes AI companies truly understand how their own systems work, and warns that AI coding agents may become the first victims of their own underlying architecture.

Confirmed

• External security researchers discovered an attack method allowing third parties to distill their models (involving an "encrypted chain-of-thought extraction" vulnerability) and responsibly disclosed it to frontier model labs.
• The frontier model providers confirmed the issue upon receiving the report, but did not publicly acknowledge it, nor did they offer any security bounty or substantial resource support. Green considers this disregard for security vulnerabilities extremely egregious.

Why it matters

• Green emphasizes that what is truly concerning is not just the inexplicability of LLMs themselves, but that code generated by AI coding agents could become the first victims of their own underlying architecture.
• The public and industry, when examining these trillion-dollar valuation labs, often wrongly assume they fully understand the workings of their black-box systems. This blind trust may pose security risks to the underlying architecture.

• Cryptographer Condemns Frontier Labs for Ignoring Encrypted CoT Extraction Vulnerability — matthew_d_green · 2026-08-11
• Security Researcher Slams Frontier Labs for Ignoring Model Distillation Attack Reports — matthew_d_green · 2026-08-11
• Crypto Expert Slams Trillion-Dollar AI Labs for Ignoring Model Distillation Attack Research — matthew_d_green · 2026-08-12
• Cryptographer Matthew Green: LLM Providers Likely Don't Understand Their Own Systems — matthew_d_green · 2026-08-12
• Cryptographer Warns AI Coding Agents Are Compromising Their Own Infrastructure — matthew_d_green · 2026-08-13

Episode 4 · European Researchers Crack Encrypted CoT of Top LLMs (2026-08-13, 5 posts)

A European research team published a paper on August 10 confirming security vulnerabilities in the hidden Chain of Thought (CoT) encryption mechanisms of mainstream frontier LLMs (such as flagship models from OpenAI, Anthropic, and Google). These encrypted CoTs can be fully extracted, raising industry concerns over AI security and data privacy.

已确认

• 漏洞机制: The issue does not lie in the encryption algorithm itself being broken, but rather that the API exposes response metadata (such as the always-available ground truth token count). Researchers point out that when the extracted token count matches the actual billed thinking token count from the API (nearly a 1:1 correspondence), it confirms with a very high probability that the real Chain of Thought has been successfully stolen.
• 跨模型迁移: Attackers can directly inject the encrypted reasoning processes extracted from advanced models (like Opus and Sonnet) into the guardrails of other models, achieving cross-model CoT transfer for jailbreaks.
• 附带数据泄露: In addition to cracking the CoT, researchers also extracted 62 real API keys from 6,708 publicly available AI session logs.

为什么重要

• Hidden CoT is a critical safety guardrail used by frontier LLM providers to protect model reasoning capabilities and prevent malicious jailbreaks. The fact that it can be extracted at a low cost and transferred across models means that existing API metadata exposure mechanisms have become a systemic security vulnerability, directly threatening the trade secrets and security defenses of top-tier AI models.

• Study Confirms: API Vulnerabilities Expose Hidden CoT in Frontier Models, Enabling Cross-Model Transfer — gsarti_ · 2026-08-13
• Stolen LLM Reasoning: OpenAI, Anthropic, and Google Share the Same Vulnerability — HuskyTheSniffer · 2026-08-13
• Researchers Extract Hidden CoT from Top LLMs and Leak 62 API Keys — xiaohu · 2026-08-13
• Encrypted Chain-of-Thought of Top AI Models Cracked, Leaking 62 API Keys — xiaohu · 2026-08-13
• Study Reveals Metadata Exploit to Steal LLM Hidden Chain-of-Thought — npinto · 2026-08-13