Preserving plaintext CoTs will look like an alchemical era of observability, argues loop transformer architect
bookwormengr · x · 2026-09-11
tszzl, architect of the loop transformer, argues that the current effort to preserve plaintext chain-of-thought as a form of observability will one day look like an alchemical era — we could do much better by understanding the model's alien ontology from the ground up. A reply quips the message to OpenAI's security folks as "no CoT, no distillation," highlighting the tension between CoT monitoring and distillation.
More from AGI Musings
- What do AI safety people actually do all day? A sincere question sparks debate — AaronBergman18 · 2026-09-11
- What do alignment researchers actually do all day? An X thread asks AI safety folks to explain — AaronBergman18 · 2026-09-11
- API resellers are an unstoppable gray market, argues virushuo on AI restrictions — tinyfool · 2026-09-11
- Qwen's Justin Lin: arch complexity hides issues evals can't catch, but agentic gains may be worth it — JustinLin610 · 2026-09-11
- Mathematicians urged to boycott AI labs: no benchmarks, no reviews, no free endorsements — sytelus · 2026-09-11
- Beyond task completion: measuring an AI agent's judgment on which experiments to abandon — VraserX · 2026-09-11