Ai2 Open-Sources Olmo-core 3 for Trillion-Parameter MoE Training

Ai2 open-sourced Olmo-core 3, a MoE training system targeting trillion-parameter models with only 5% throughput drop at 128 experts, alongside a technical report and interactive walkthrough of 512-GPU training.

2026-10-01 ~ 2026-10-01 · 3 related posts

• Ai2 open-sources Olmo-core 3, training infrastructure scaling MoE models toward trillion parameters — allen_ai · 2026-10-01
• Ai2's tech report shows how 512 GPUs train one model—and flags 'token gerrymandering' — allen_ai · 2026-10-01
• Ai2 Releases Olmo-core 3: Open MoE Training Framework Scales to 128 Experts with <5% Throughput Loss — allen_ai · 2026-10-01