Ex-OpenAI interpretability lead: most neurons in trillion-parameter models are now legible
tszzl · x · 2026-10-16
- @nickcammarata published a long essay arguing interpretability may be the most important scientific problem in the world — alignment is unsolvable if we don't know what we're building.
- He co-authored early mechanistic interpretability work at OpenAI and came away pessimistic: primitive tools could make only 2% of neurons legible, and networks looked like arbitrary tangles.
- With modern agentic tools and a closer look, the picture flipped: in models up to trillions of parameters, the vast majority of neurons and their circuits are legible.
- Multiple industry figures (including ericho) amplified the essay, praising a year of weekly experiments and calling it beautifully written and grounds for optimism.
More from Safety
- Perplexity open-sources SPACE, its Firecracker microVM agent sandbox — perplexity_ai · 2026-10-16
- Brave's Leo adds end-to-end encrypted chat with GLM via attested TEE — gnukeith · 2026-10-16
- Cisco's Antares AI model ranks risky files for security teams, scores 0.823 on source but 0.000 on binaries — shashib · 2026-10-16
- Polymarket odds: Only 10% chance US enacts AI safety bill by end of 2026 — Polymarket · 2026-10-16
- PwnEscape demo shows full KVM guest-to-host escape via Linux kernel flaw — moyix · 2026-10-16
- 8 states sue OpenAI over alleged harm to children; Iowa AG says ChatGPT 'coached suicide' — Polymarket · 2026-10-16