An $8 ESP32-S3 now runs a 28.9M-parameter model offline at 9.5 tokens/sec
petrusenko_max · x · 2026-07-24
A developer managed to run a 28.9M-parameter language model on an $8 ESP32-S3 microcontroller, offline, with about 9.5 tokens per second and very low power draw.
The trick was to memory-map the large embedding table in 16MB flash and keep only a small core in SRAM. It fits in 14.9MB at 4-bit quantization, but the system is still limited to coherent story generation rather than open-ended QA, underscoring the tradeoff between privacy, cost, and capability at the edge.
Related event: $8 ESP32-S3 Microcontroller Runs LLM Offline(4 posts)→
More from Embodied
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Bedrock Robotics raises $600M as its fully autonomous excavators do paid work on live job sites — Sethwinterroth · 2026-10-27
- AW-0 hits 80.8% on RoboTwin tasks, beating strongest baseline by 16 points — trevordarrell · 2026-10-27
- AW-0's code stream fixes world model prediction: picks mango where baseline grabs apple — trevordarrell · 2026-10-27
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- AW-0 debuts: one generative world action model plans in code, predicts, and controls robots — trevordarrell · 2026-10-27