Nemotron 3.5 Bolt E4B hits 27.7 tok/s on Jetson Nano, 1.8x faster than llama.cpp
Course_Latter · reddit · 2026-11-05
The embedl team got NVIDIA's latest small model, Nemotron 3.5 Bolt E4B, running at 27.7 tok/s on a Jetson Nano (Orin Nano Super) — 1.8x faster than llama.cpp — fully on-device, enough to code small games. They host Jetson Nanos serving the model on their website so anyone can feel the real on-device performance.
More from Infra
- WSJ: OpenAI leaders weigh resetting data center expectations amid AI bubble worries — SumitGup · 2026-11-05
- Hob: free open-source local AI app that reads docs and fills PDFs fully offline — Div_pradeep · 2026-11-05
- Vertex AI users report Google secretly breaking KV cache, driving up token bills — niwatoriking · 2026-11-05
- Building a $2,000 CAD ComfyUI PC: used RTX 3090 24GB vs newer GPUs? — Good_Business7425 · 2026-11-05
- OpenAI reportedly to cover full power costs of future data centers — inductionheads · 2026-11-05
- Dropbox acquires Tigris Data for $350M to bolster its AI infrastructure — TechCrunch AI · 2026-11-05