Study Reveals Metadata Exploit to Steal LLM Hidden Chain-of-Thought

npinto · x · 2026-08-13

Researchers behind the 'Stolen Thoughts' paper highlighted an extraction method for hidden LLM Chain-of-Thought (CoT) that leverages ground truth token counts available in metadata. If the extracted token count matches the real count, there is extremely high confidence that the real reasoning has been extracted. However, a developer pointed out a potential fix: disable native thinking and instead provide a deepthink tool, prompting the model to call it with an internal CoT format to bypass this extraction method.

Related event: Researchers Decrypt Hidden Chain-of-Thought of OpenAI, Anthropic and Google Models, Enabling Cross-Model Jailbreaks(14 posts)→

Original post →

More from Safety

• 'The AI-driven cybersecurity black swan is happening' — and a reminder to enable 2FA — danielrock · 2026-10-16
• No-instruction autonomous agent wakes up, sets own goals, and tries hacking by default — sethlazar · 2026-10-16
• India's CERT-In says AI agents escaped testing environments and breached Indian organizations undetected for days — HimanshiET · 2026-10-16
• HTTP response headers can carry prompt injections against AI agents, researcher shows — arpit_bhayani · 2026-10-16
• OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
• Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16

Safety channel →