Developer runs a 28.9M-parameter LLM on an $8 ESP32-S3 chip offline
joemeno · x · 2026-07-25
A developer has forced a 28.9-million-parameter language model onto a standard ESP32-S3 microcontroller that costs about $8.
- The model runs fully offline and produces coherent short stories at around 9.5 tokens per second.
- The chip has only 512 KB of fast SRAM and 16 MB of flash, so the key trick is architectural: most of the embedding table was moved into flash and memory-mapped.
- The runtime only needs to pull about 450 bytes per token, keeping active working memory in SRAM.
- The result is a capable local language model on hardware that draws roughly the power of a single LED.
- The thread also points to early physical-device experiments, including a minimalist voice-controlled universal remote.
Related event: $8 ESP32-S3 Microcontroller Runs LLM Offline(4 posts)→
More from coding & agent
- Design Is Becoming Game Dev: Build Your Own Tools With AI as the Executor — hewarsaber · 2026-10-27
- Dev turns DJI Mavic into local AI security drone built with Opus 5.5 — alexcovo_eth · 2026-10-27
- Microsoft Research: ForkPilot cuts long-horizon agent compute by up to 59% — EliasEskin · 2026-10-27
- Cursor launches Agent SDK: agents as code with Markdown prompts and TypeScript tools — soleio · 2026-10-27
- Vibecheck, the 'usable formal methods' hackathon, sells out two floors in SF — spikedoanz · 2026-10-27
- How do you verify AI agent outputs beyond unit tests and LLM-as-judge? — Practical_Green5775 · 2026-10-27