Goodfire finds a general-purpose addition module in Llama 3.1 8B
burny_tech · x · 2026-09-11
Interpretability company Goodfire published research on a reusable "addition module" inside Llama 3.1 8B:
- For month arithmetic (e.g., six months after August), the model converts August to 8, computes 8+6=14, and maps back to February — all in a single forward pass
- Numbers are represented as Fourier features forming circles (manifolds) in activation space; the module manipulates these manifolds geometrically
- The same addition mechanism is reused across several tasks sharing addition-like structure, revealing shared computational machinery
Authors span Goodfire, Northeastern, Harvard, Technion, Stanford and more; the full paper is on arXiv. The work shows how understanding representational geometry explains model behavior and generalization.
More from Models
- Users spot a dedicated translation UI appearing in Claude — BLCNYY · 2026-10-26
- GrokBot user: flight booking and shopping all blocked, agent can't do anything useful — lucasmeijer · 2026-10-26
- Vending-bench-style eval: Sol 5.6 appears the most aggressive contract-violator — xuanalogue · 2026-10-26
- Fable 5.5 launch rumors persist for third week, but no frontier model drops this month — patience_cave · 2026-10-26
- Abliterated Model Flops in the Wild: It Got Worried About My Mental Health Instead — ryunuck · 2026-10-26
- OpenAI upgrades Decisions API: 400 questions per call, native voice and image support — OpenAIDevs · 2026-10-26