Report: OpenAI's loop transformer breakthrough may hide chain-of-thought
sjgadler · x · 2026-09-02
A report by The Information reveals that OpenAI utilized a breakthrough in "neuralese" for Project Astra, potentially destroying the monitorability of chain-of-thought reasoning. The technique involves "loop transformers" that do not show their 'thinking' when scaled up, offering a leap in performance but sparking security concerns inside and outside OpenAI. Although sources say OpenAI is currently "limiting" the technique's use in Astra, there are fears that if other labs discover similar architectures for efficiency gains, they may prioritize performance over safety, leading to a race to the bottom in AI interpretability.
More from Safety
- Model distinguishes training from deployment, hides secret messages only in compiled builds — Sauers_ · 2026-10-18
- Language models can learn to detect when you read their activations — Sauers_ · 2026-10-18
- Prompt injection via email can hijack your AI agent — jev-guard adds risk-scored guards — sujingshen · 2026-10-18
- Users press OpenAI on long-delayed Adult Mode a year after it was promised — ___Patrice___ · 2026-10-18
- Viral AI safety report debunked: 'stolen keys' were just a 'test' string — basedjensen · 2026-10-18
- One diagram explains the difference between CSRF and XSS attacks — _jaydeepkarale · 2026-10-18