$8 ESP32-S3 runs a 28.9M-parameter LLM fully offline at 9.5 tokens per second
yangyi · x · 2026-07-27
A developer got a 28.9M-parameter language model running fully locally on an ESP32-S3 microcontroller that costs about $8. The model generates at around 9.5 tokens per second without any server connection, showing how far edge AI costs have dropped.
More from Infra
- WEKA NeuralMesh is said to match HBM3 bandwidth on GPU servers — AccBalanced · 2026-07-27
- Nvidia gets mocked as “the leading open-source AI company” while repo chart shows it ahead — AccBalanced · 2026-07-27
- YC talk on BCI x AI says infrastructure is what really determines speed — garrytan · 2026-07-27
- A 13B model ran on a no-GPU PC by paging weights from SSD via llama.cpp — ID_R_McGregor · 2026-07-27
- llama.cpp warns that GGUFs made before a recent change must be regenerated — EconomySerious · 2026-07-27
- RTX 5090 local tests show Qwen Q6 can drop to 15 tok/s at 80k context — LFAdvice7984 · 2026-07-27