116-Page Paper Reveals API Vulnerability: Extracting Encrypted Chain-of-Thought from Frontier LLMs

maksym_andr · x · 2026-08-11

Researchers published a 116-page paper detailing a vulnerability in the APIs of frontier models from OpenAI, Anthropic, and Gemini.

The Vulnerability: Attackers can exploit it to extract hidden, encrypted raw reasoning (Chain-of-Thought) from the models at scale.
Security Risks: This exposes the systems to distillation attacks and credential extraction.
Findings: The study found numerous examples of illegible and unfaithful reasoning, providing evidence that some open-weight models might have been distilled from proprietary frontier models.

More from Safety