Viral ESP32 AI Project Runs 28.9M Parameter LLM Fully Offline
minchoi · x · 2026-08-04
Developer slvDev has open-sourced their ESP32 micro-AI project on GitHub, quickly gaining over 3,400 stars. The project successfully runs a 28.9 million parameter language model natively on an ESP32-S3 microcontroller.
By leveraging the Per-Layer Embeddings technique from Google's Gemma 3n, the project stores the vast majority of parameters in flash memory. This enables fully offline local inference on a chip with only 512KB of SRAM, achieving an end-to-end generation speed of 9.88 tokens/sec.
Related event: Developer Runs 28M Parameter AI on $8 ESP32 Microcontroller(2 posts)→
More from Infra
- YC-Backed Lamb Labs Claims 63x Higher Efficiency for AI Inference Chips vs GPUs — ycombinator · 2026-08-05
- DSpark Open-Sources Speculative Decoding Path for Kimi K3 — ying11231 · 2026-08-05
- US Drafts Ban on Chinese Datacenter Components as Europe Pushes for Tech Sovereignty — nordicinst · 2026-08-05
- Cursor Releases Mixture-of-Kittens Megakernel for MoE, Claims Nearly 2x TFLOP/s — CapnHat · 2026-08-05
- NVIDIA Tutorial: Building Fully Local Autonomous Agents on Jetson — NVIDIA Developer · 2026-08-05
- YC Launches Caution Hosting: Secure Enclaves for Sensitive AI Workloads — ycombinator · 2026-08-05