An $8 ESP32-S3 now runs a 28.9M-parameter language model offline
brianrkelly · x · 2026-07-24
- A developer has forced a 28.9-million-parameter language model onto an ESP32-S3 microcontroller that costs about $8.
- The model runs fully offline, generates short stories at roughly 9.5 tokens per second, and uses power comparable to a small LED.
- The poster frames it as a dramatic expansion of what “local AI” can mean on tiny hardware.
- He also suggests a network of many such devices could specialize on different domains and report back to a master node.
Related event: $8 ESP32-S3 Microcontroller Runs LLM Offline(4 posts)→
More from Infra
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- NVIDIA Claims HGX Rubin NVL8 Delivers 10x More Tokens per Megawatt Than B200 — thursdai_pod · 2026-10-27
- Commoditization dilemma: either human labor or GPUs become the commodity — _AustinCalvert_ · 2026-10-27
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Running a local 9B as a coding agent: brief the big model to plan, let the small model execute — Electrical_Ninja3805 · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27