Two-person team trains HobbyLM MoE in ~304 GPU-hours, 100-400x less compute than peers
msharmas · x · 2026-10-15
Fuellabs, a two-person team with no research lab, released HobbyLM: a sparse MoE trained from scratch with 1.037B total parameters (305M active per token) on 100B tokens. The full base run took 76 hours on 4 H200s — roughly 304 GPU-hours, versus 122,880 for Sarvam-1, 153,600 for Zoho Zia 7B, 370K for Llama 3.2 1B, and 460K for Llama 3.2 3B (100-400x fewer). Engineering decisions and failed experiments to be shared in follow-ups.
More from Infra
- llama.cpp tuning alone makes Qwen3-4B 1.34x faster on a 2019 phone, offline — hbouammar · 2026-10-15
- a16z chart: old GPUs are appreciating assets as B200 rental hits $5.69/hour amid compute scarcity — a16z · 2026-10-15
- 1GW of Compute Matches the Gross Profit of 1M Cybercabs — SpaceX to Add 3-5GW Next Year — JOBhakdi · 2026-10-15
- Cloudflare data: Yandex outage didn't kill malicious bots, they just moved to other Russian hosts — Cloudflare Blog · 2026-10-15
- AI cost falling 54x faster than electricity ever did, five years in — LinusEkenstam · 2026-10-15
- Inference has no yardstick: 4.5-quadrillion token run rate, but no proof of what buyers get — AccBalanced · 2026-10-15