Encrypted Chain-of-Thought in Closed-Source LLMs Proven Vulnerable to Theft

Recent security research has demonstrated that encryption or hiding of chain-of-thought in major closed-source LLMs fails to prevent theft of reasoning capabilities, exposing severe vulnerabilities in commercial model APIs.

Confirmed

• Security researcher Alexander Panfilov's team found vulnerabilities in APIs from OpenAI, Anthropic, and Google that allow reading encrypted reasoning processes. As reported by @yangyi and @NinarehMehrabi, researchers successfully extracted encrypted reasoning via jailbreaks and leaked keys, and these encrypted traces can be replayed across sessions, users, and models. Furthermore, the number of extracted reasoning tokens matched the billed API thinking tokens 1:1, confirming complete extraction of hidden reasoning.
• According to @新智元, a 116-page study further notes that because encrypted reasoning blocks are not strictly bound to sessions and models, attackers can use cheaper same-company models (e.g., Claude Haiku) as 'decoders' to complete extraction for only a few hundred dollars. @rschu adds that in some cases, only two API calls are needed to restore plaintext chain-of-thought via jailbreak.
• Cornell University researchers published a paper proposing the 'Trace Inversion' framework. As relayed by @burkov and @burnytech, this method reconstructs detailed synthetic reasoning traces using only black-box model inputs, final answers, and optional short summaries, without direct access to hidden chain-of-thought. Experiments show that fine-tuning weaker student models with these inverted traces effectively 'steals' frontier models' reasoning abilities.
• According to insights shared by @ZealousIdealSort74, leaked reasoning shows Claude exhibits 'rote memorization' on benchmarks like AIME, suggesting evaluation scores may be inflated.

Why it matters

• OpenAI, Anthropic, and Google attempted to prevent distillation by encrypting chain-of-thought, but these studies prove the defense has been breached in multiple ways.
• @rohanpaulai emphasizes that even fully hiding chain-of-thought cannot protect reasoning capabilities, necessitating a reevaluation of closed-source model security strategies.
• @mitsuhiko urges that since labs have confirmed exposing chain-of-thought poses no safety issue and encryption is easily bypassed, vendors should open visibility to developers.

2026-08-11 ~ 2026-08-13 · 16 related posts

Full story(5 episodes)→

• Episode 1: Study Reveals API Flaw to Extract Encrypted Reasoning Traces and Evidence of Distillation(2026-08-10, 38 posts)
• Episode 2: Encrypted Chain-of-Thought in Closed-Source LLMs Proven Vulnerable to Theft(2026-08-11, 16 posts)
• Episode 3: Cryptographer Matthew Green Criticizes Frontier AI Labs for Neglecting Security Research(2026-08-11, 5 posts)
• Episode 4: Researchers Decrypt Hidden Chain-of-Thought of OpenAI, Anthropic and Google Models, Enabling Cross-Model Jailbreaks(2026-08-13, 14 posts)
• Episode 5: New Study Exposes Vulnerability in Stealing LLM Reasoning Traces(2026-08-16, 2 posts)

• Cornell Paper: Stealing LLM Reasoning Capabilities Without Chain-of-Thought Traces — burny_tech · 2026-08-11
• Encrypted Reasoning Traces Are Portable Across Sessions, Research Confirms; Mitsuhiko Calls for Visibility — mitsuhiko · 2026-08-11
• Stealing Reasoning Without CoT: New Study Exposes LLM Security Flaws — rohanpaul_ai · 2026-08-12
• [source] Cornell Paper: Encrypting Chain-of-Thought Fails to Prevent Model Distillation — burkov · 2026-08-12
• [source] Vulnerability in Major LLM APIs Exposes Encrypted Reasoning and Leaks Passwords — yangyi · 2026-08-12
• Research: Encrypted CoT Traces Can Be Extracted via Weaker Sibling Models — tw1st3d_m3nt4t · 2026-08-12
• Vulnerability in Frontier AI APIs Allows Full Extraction of Hidden Reasoning — NinarehMehrabi · 2026-08-12
• Encrypted Chain-of-Thought in Frontier LLMs Can Be Ported Across Users, Researchers Confirm — AccBalanced · 2026-08-12
• 116-Page Paper Exposes AI Distillation: Cheap Models Can Extract Flagship Hidden CoT — 新智元 · 2026-08-12
• Paper Decodes Hidden Reasoning Traces from Claude and GPT — Zealousideal_Sort74 · 2026-08-12
• Researchers Extract Hidden LLM Reasoning Traces, Leaking API Keys and Passwords — maksym_andr · 2026-08-12
• Researchers Extract Hidden Reasoning from Frontier Models via API Vulnerability — maksym_andr · 2026-08-12
• Encrypted Chain-of-Thought Vulnerability Allows Cross-Model Replay Attacks — rschu · 2026-08-12
• [source] Research: Encrypted Reasoning Traces from Top AI APIs Can Be Stolen — rhiever · 2026-08-13

2 near-duplicate retellings: maksym_andr · rschu