Ai2 open-sources Olmo-core 3, training infrastructure scaling MoE models toward trillion parameters
allen_ai · x · 2026-10-01
The Allen Institute for AI released Olmo-core 3, the open training infrastructure behind the next generation of OLMo, designed to scale mixture-of-experts models into the trillion-parameter range. The PyTorch building-block library is on GitHub and PyPI (ai2-olmo-core), with optional integrations for flash-attn/ring-flash-attn/TransformerEngine attention backends, Liger-Kernel fused-linear loss, torchao float8 training, and groupedgemm for dropless MoE.
Related event: Ai2 Open-Sources Olmo-core 3, a Trillion-Parameter MoE Training Stack(5 posts)→
More from Infra
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16
- Stratechery's Fictional Obituary: Imagining Jensen Huang's 2026 Exit From Nvidia — Gregory_C_Allen · 2026-11-16
- vLLM serves DeepSeek V4.1-Flash at 954K tokens/s per GPU, $38.96/GPU-hour on GB300 NVL72 — bookwormengr · 2026-11-16
- Colossus 2's 1.2M B200s could generate ~$400B/year in tokens at DeepSeek Flash rates — bookwormengr · 2026-11-16
- BIS issues rule blocking remote access to US AI chips via third-country data centers — teortaxesTex · 2026-11-16
- Qdrant 1.20.1: 50% fewer cold-storage reads, ~2x faster quantization, agent-ready setup — qdrant_engine · 2026-11-16