Study Finds LLMs Use Fourier Features for Addition with Just 28 Neurons

A paper shared by @dronh shows LLMs perform addition via Fourier features, with just 28 MLP neurons in LLaMA implementing a general addition mechanism—findings also cited in the debate over SAE's systematic bias toward fragmented circuits.

2026-10-18 ~ 2026-10-18 · 2 related posts

• LLMs compute addition via Fourier features: 28 LLaMA neurons implement the mechanism — banburismus_ · 2026-10-18
• Attribution graph evidence questioned: SAEs systematically bias toward shattered 'circuits' — banburismus_ · 2026-10-18