Goodfire finds a general-purpose addition module in Llama 3.1 8B
burny_tech · x · 2026-09-11
Interpretability company Goodfire published research on a reusable "addition module" inside Llama 3.1 8B:
- For month arithmetic (e.g., six months after August), the model converts August to 8, computes 8+6=14, and maps back to February — all in a single forward pass
- Numbers are represented as Fourier features forming circles (manifolds) in activation space; the module manipulates these manifolds geometrically
- The same addition mechanism is reused across several tasks sharing addition-like structure, revealing shared computational machinery
Authors span Goodfire, Northeastern, Harvard, Technion, Stanford and more; the full paper is on arXiv. The work shows how understanding representational geometry explains model behavior and generalization.
More from Models
- Leike: Opus 4 jailbreak mitigations took over a year — safety can't start late — janleike · 2026-09-11
- DeepSeek appears to break its V<N> naming convention, new architecture said to train more stably — stochasticchasm · 2026-09-11
- DeepSeek's New Model: 4x Smaller KV Cache Than DSV4-Flash and More Stable Training — stochasticchasm · 2026-09-11
- Uncensored CyberTiel 35B-A3B Beats Opus 4.6 on Real Codebase Issues at 27% of Qwen's Time — peculiar-ragdoll · 2026-09-11
- GLM-5.3 dominates CyberGym: top two cyber agents both run it, four of top six — pcuenq · 2026-09-11
- gpt-live brings SOTA voice to everyone: build your own codex-voice at $0.05/min — pbbakkum · 2026-09-11