Meta team trains 30B LLM from scratch in months, squeezes it into a 32GB GPU

Meta engineer TimDarcet shared that he and colleagues including Parth, Kelvin, and Anton spent a few months building a team and a complete training pipeline, training an LLM called Muse Glimmer-30B from scratch. He described the results as "quite good" and sees it as a promising starting point for a "second era" of Meta open source. Although the model fits in VRAM, memory bandwidth and throughput remain bottlenecks, so the team tackled deployment via two routes: quantization and speculative decoding.

Confirmed

• The model is Muse Glimmer-30B, trained from scratch by a small team within a few months.
• The EvoPress quantization scheme was obtained via evolutionary search: average accuracy 75.6% (vs. 75.7% for bf16), average response length 11417 (vs. 11070 for bf16), with weight size reduced from 56GB to 19GB—nearly lossless.
• The quantization scheme was found automatically through greedy and evolutionary search, jointly minimizing VRAM usage and perplexity error, yielding two Pareto-optimal trade-off points; TimDarcet confirmed the results are Pareto-optimal.
• Equipped with a DFlash speculative decoding drafter: 5-layer draft, block size 16 (predicting 16 tokens at a time), to relieve memory bandwidth and throughput pressure.
• The end result: a 30B-parameter model that runs on a 32GB GPU.

Why it matters

This work demonstrates a small team completing a full-chain practice—from data to training, quantization, and inference acceleration—within a few months. In particular, the combination of search-based quantization and speculative decoding offers a referenceable path for deploying mid-scale models on consumer GPUs. giffmana also relayed the quantization research approach and jokingly asked why agentic search wasn't used, indirectly reflecting how much attention this direction is attracting.

2026-10-21 ~ 2026-10-21 · 5 related posts

Full story(2 episodes)→

• Episode 1: Meta Open-Sources Muse Glimmer 30B, an On-Device Agent Model(2026-10-20, 5 posts)
• Episode 2: Meta team trains 30B LLM from scratch in months, squeezes it into a 32GB GPU(2026-10-21, 5 posts)

• [source] How a 30B model fits on a 32GB GPU: search-based quant plus spec decoding — TimDarcet · 2026-10-21
• [source] EvoPress quantization shrinks 30B model from 56GB to 19GB at near-bf16 accuracy — TimDarcet · 2026-10-21
• DFlash drafter for Muse Glimmer-30B: 5 layers, blockwise 16-token spec decoding — TimDarcet · 2026-10-21
• [source] Meta team builds open-source 30B LLM from scratch in months, hails 'second era' — TimDarcet · 2026-10-21
• Custom quantization scheme found via greedy and evolutionary search, Pareto optimal — giffmana · 2026-10-21