Ai2 ships Olmo-core 3 training framework scaling MoE LLMs past 1 trillion parameters
dl_weekly · x · 2026-10-13
The Allen Institute for AI released Olmo-core 3, a training framework that makes large mixture-of-experts (MoE) LLM training far more efficient.
Key points:
- Scales the expert pool from 8 to 128 while activating only 4 experts per token;
- Supports training models beyond 1 trillion parameters on the same infrastructure;
- Benchmarked at 52,000 tokens per second for a 47B-parameter model on Nvidia B3000 GPUs;
- Reduces expert-routing and networking overhead, closing the gap between dense and MoE training compared with Nvidia's Megatron-core architecture.
More from Infra
- Intel's Crescent Island GPU lands in D-Series lineup with up to 480GB of LPDDR5X — Monad_Maya · 2026-10-13
- NVIDIA chief scientist headlines silicon-to-systems panel at Applied Materials Epic launch — BenBajarin · 2026-10-13
- Zuckerberg: Owning data centers is a financial edge as depreciated H100s gain value — BrentM_BuildGuy · 2026-10-13
- NVIDIA's LoGRA cuts RL training memory by 45.7%, using less than LoRA — omarsar0 · 2026-10-13
- Building a GPU compiler from scratch: lowering PyTorch to CUDA via six IRs — NoVibeCoding · 2026-10-13
- The Neocloud Map: physical ownership vs platform depth are separate axes — AccBalanced · 2026-10-13