Ex-OpenAI interpretability lead: most neurons up to trillions of params are now legible
josh_wills · x · 2026-10-17
nickcammarata, who co-authored early mechanistic interpretability work at OpenAI and launched its first large-scale automated interpretability project, argues interpretability may be the most important scientific problem today. Primitive tools once made only 2% of neurons legible; with modern agentic tools, the vast majority of neurons in models up to trillions of parameters are legible. In a reply, Dorialexander adds an overlooked factor: data shape — models like Qwen-3 and Kimi are trained on vast amounts of clean synthetic data, which affects superposition and feature formation.
More from Models
- KOL teases mystery model argon, says people are 'severely underestimating' it — iruletheworldmo · 2026-10-17
- Perplexity decider v1.2 lands #3 on Decision Index as longer training erodes backbone skills — antoine_chaffin · 2026-10-17
- Perplexity's 4B decider model distills from its 27B sibling using the same training recipe — antoine_chaffin · 2026-10-17
- Perplexity traces decider v1.2 regressions to longer training, mitigates via model souping — antoine_chaffin · 2026-10-17
- Perplexity details decider data hygiene: official splits only, deduped, 16.6% of one stage — antoine_chaffin · 2026-10-17
- Perplexity says rivals train on Decision Index benchmark sets, opts to follow for fair comparison — antoine_chaffin · 2026-10-17