smolbenchmark ranks sub-8GB models by speed, tok/J and heat on your own hardware
East-Muffin-6472 · reddit · 2026-09-12
- Mainstream leaderboards assume server GPUs; smolbenchmark targets models that fit in 8GB, ranked by decode speed, tokens per joule, and heat.
- Covers tablets, phones, Macs, Jetsons and Raspberry Pis with 13 model families so far and 1,000 configs on the Jetson nano Orin Super 8GB; a live device streams tok/s, tok/J, ITL, latency, power, thermals and battery.
- All raw data and reports are public; Pi, phone and Mac mini numbers are still being filled in, and feedback is welcome.
More from Infra
- effect-harness on Cloudflare: run agent loops, tools and credentials on separate VMs — samgoodwin89 · 2026-10-31
- Google's Project Suncatcher boots TPU in orbit and beams back first AI response — sundarpichai · 2026-10-31
- Together AI launches Post-Training platform for agentic RL with tool-use environments — togethercompute · 2026-10-31
- General Compute raises $100M for inference capacity as CoreWeave stock hits 12-month low — YvesMulkers · 2026-10-31
- Analyst warns of unstable AI compute feedback loop as Kioxia's NAND price warning triggers flash crash — bookwormengr · 2026-10-31
- SOMA launches enterprise prompt-compression layer claiming 20%+ inference savings — markjeffrey · 2026-10-31