Meta team builds open-source 30B LLM from scratch in months, hails 'second era'
TimDarcet · x · 2026-10-21
Meta engineer TimDarcet recounts how a small team built a full training pipeline and an LLM from zero in a few months, calling it a promising start for a 'second era of open-source at Meta.'
Follow-up posts reveal the model, Muse Glimmer-30B, runs on a single 32GB GPU thanks to EvoPress — a custom quantization scheme found via greedy + evolutionary search that shrinks weights from 56GB to 19GB at near-bf16 accuracy — plus a DFlash speculative-decoding drafter (blockwise, 16 tokens). Benchmarks vs Gemma4-31B and Qwen3.6-27B show wins on SWE-Bench Pro (51.2), AIME 2026 (94.7), multilingual evals and privacy (CIMemories).
Related event: Meta Engineers Train a 30B LLM from Scratch and Squeeze It onto a 32GB GPU(4 posts)→
More from Companies & People
- AI creator Bilawal Sidhu crosses 1M followers on TikTok with 9.8M likes — bilawalsidhu · 2026-10-21
- Jensen Huang tells German Chancellor Merz: nothing to regulate if the industry doesn't thrive — rohanpaul_ai · 2026-10-21
- Jensen Huang: every company will have two factories, one real and one digital — round · 2026-10-21
- Palo Alto, not SF, is now the hottest neighborhood for new AI labs — infoxiao · 2026-10-21
- Ex-Twitter engineer credits X's growth to the long-delayed Android rewrite — Scobleizer · 2026-10-21
- Walden Robotics CTO joins Long Horizon panel on large behavior models with Rodney Brooks — adnothing · 2026-10-21