28.9M-parameter LLM runs on $8 ESP32-S3 at 9.5 tokens/s, no cloud needed
thehiphopswami · x · 2026-08-02
The open-source project esp32-ai deploys a 28.9M-parameter language model on the ESP32-S3 microcontroller, which costs about $8. It runs entirely on-device, generating text on a small screen at roughly 9.5 tokens per second, with no cloud or API calls. This is about 100x more parameters than previous models on similar chips (260K), achieved by storing most of the model in flash using Per-Layer Embeddings from Google's Gemma. The project has gained 2.8k stars on GitHub, showcasing new possibilities for AI hardware, potentially enabling AI toys, robots, and IoT devices without cloud dependency.
Related event: Developers Run 29M Parameter LLM Locally on $8 ESP32 Microcontroller(3 posts)→
More from Embodied
- Peking University's SimForcing distills simulation motion priors into robot world models — PekingUniversity · 2026-10-16
- NVIDIA and Mercedes-Benz Expand Partnership on Next-Gen Automated Driving — nvidia · 2026-10-16
- ~25,000 Humanoid Robots Shipped in H1 2026, Over 95% from Chinese Vendors — ingliguori · 2026-10-16
- openpilot brings stoplight-stopping autonomy to a 2019 Corolla — k7agar · 2026-10-16
- Hironobu Fujiyoshi publishes deck: from AI agents to Agentic Robotics — HirokatuKataoka · 2026-10-16
- CoDA co-evolves simulation and robot policies via shared failure memory, lifting OOD success from 5.3% to 27.6% — PLAN-Lab · 2026-10-16