28.9M parameter LLM runs entirely on ESP32-S3 microcontroller
tom_doerr · x · 2026-08-20
A developer successfully runs a 28.9 million parameter LLM on an ESP32-S3 microcontroller.
Technical Details:
- Storage: 28.9M parameters total, with 25M stored in a Flash lookup table (using Per-Layer Embeddings from Google's Gemma 3n) to fit within limited SRAM.
- Hardware: ESP32-S3, 512KB SRAM, 8MB PSRAM, 16MB Flash.
- Performance: 9.88 tok/s end-to-end, 94.9 ms/token compute time.
- Independence: Runs entirely on-device with no connectivity.
- Size: 14.9MB at 4-bit quantization.
The project demonstrates that LLMs can run on resource-constrained microcontrollers by storing weights in Flash instead of RAM.
More from Embodied
- Guide to running local vision on ESP32-S3 with camera integration — blaizedsouza · 2026-08-20
- MIT Press open-sources 'Introduction to Autonomous Robots' textbook covering kinematics to neural networks — blaizedsouza · 2026-08-20
- Starkey claims nearly a decade of AI investment in hearing aids — BrandonSawalich · 2026-08-20
- Blueprint raises $1.4M CAD to bring vibe coding to hardware — Scobleizer · 2026-08-20
- Core robotics challenge: achieving one-shot generalization in the real world — Scobleizer · 2026-08-20
- Google's RT-2 packed 55B parameters, 1000x RT-1's 55M just 7 months later — binarybits · 2026-08-20