Meta Releases MobileMoE: Sub-Billion Active Parameter On-Device Models
jacek2023 · reddit · 2026-08-25
- Family: MobileMoE is a family of on-device Mixture-of-Experts (MoE) language models with sub-billion active parameters, designed to push the quality-efficiency frontier for mobile LLMs.
- Scales: Available in three sizes (S/M/L) with 0.3B/0.5B/0.9B active parameters (1.3B/2.8B/5.3B total).
- Efficiency: INT4 weight footprints are <3 GB to fit in mobile DRAM.
- Variants: Each scale is released in three variants: Base (pre-training + mid-training), SFT (supervised fine-tuning), and QAT (quantization-aware training).
More from Models
- New inference engine 'e' hits 10,500 tok/s on a single DGX Station — brandon_galang · 2026-10-16
- Reasonable post-trains Nemotron 3.5 on 3,000 Verus proofs, lifting pass@1 ~30 points to 73.3% — fhuszar · 2026-10-16
- NVIDIA's Nemotron 3.5 Super VL hits CoreWeave: 12B-active multimodal MoE at $0.25/M input tokens — wandb · 2026-10-16
- Opus 5 might be a very deep model or use looping, commentator speculates — scaling01 · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- Nvidia-backed post-training pushes Nemotron 3.5 VL past frontier on dermatology diagnosis — LoganGrasby · 2026-10-16