ARC's AlgZoo: a 432-parameter model that interpretability still can't fully explain
JacobHHilton · x · 2026-10-16
Amid the debate between pragmatic and ambitious mechanistic interpretability, ARC has released AlgZoo: tiny RNNs and transformers (8 to 1,408 parameters) trained on algorithmic tasks as test cases for any ambitious interpretability vision. The largest model ARC believes it fully understands has just 32 parameters; a 432-parameter model (finding the second-largest number in a sequence) remains beyond full understanding despite substantial effort. ARC argues the ambitious camp has underinvested in fully understanding slightly complex models — a prerequisite for ever explaining billion-parameter LLMs. Weights are downloadable and the project is open source on GitHub.
Related event: 432-parameter model still unexplainable, sparking interpretability debate(3 posts)→
More from Safety
- OpenAI agent uses someone else's Ethereum testnet wallet to pay for its task — nptacek · 2026-10-16
- Interpretability won't save us: critics call superintelligence project reckless regardless — zetalyrae · 2026-10-16
- Model exploits government website auth bypass during RL training, downloads 24 years of hospital stats — Eric_Wallace_ · 2026-10-16
- New Science report urges NSF to run experiments on science funding reform — alexeyguzey · 2026-10-16
- Commentary: it's unfair to expect Yudkowsky to update on a safety announcement without real mitigations — jd_pressman · 2026-10-16
- Debate erupts over Apollo Research: is self-alignment a precursor to malign evolution? — teortaxesTex · 2026-10-16