Reverse-engineering Apple's on-device 20B model: MoE in flash, only 2GiB RAM
julsimon · x · 2026-10-25
Every Apple Intelligence-capable Mac carries a 20-billion-parameter model callable from the terminal. With no official model card, Julien Simon built one from the outside via file reads, memory, power and token measurements. Key findings: the model lives in flash storage, wakes a few MoE experts per prompt on the Neural Engine, and holds only 2GiB of memory after a long request vs 15GiB for a comparable conventionally-built open model. Google contributed to training; the engineering is Apple's. All scripts and datasets are on GitHub for reproduction.
More from Infra
- Researchers rebuild city power grid topology from open data, no utility secrets needed — anselm · 2026-10-25
- 'Pls just one more GPU': the begging-for-H100 meme mocking scaling obsession — miniapeur · 2026-10-25
- SpaceX borrows another $40B for Nvidia chips as bond investors balk at space data centers — jonerp · 2026-10-25
- Musk predicts 'stranded compute' in half-built data centers, questioning infinite demand — mjdramstead · 2026-10-25
- Nvidia open-sources Flora, cuts robot video labeling cost 125x vs GPT-6 Astra — The Decoder · 2026-10-25
- What the US Movement Against Data Centers Can Learn From Abroad — Wired AI · 2026-10-25