IFM open-sources K2 Horizon: six models, 20T tokens each, and a public reward-hacking audit
kimmonismus · x · 2026-09-09
A thread by kimmonismus breaks down IFM's K2 Horizon open model family, a new bar for open-source transparency:
- Lineup: 0.9B, 3.7B, 7B, 32B, 36B-A4B and 375B-A23B; the two sparse models activate only 4B and 23B parameters per token, spanning edge devices to enterprise workloads with shared tooling
- Training: each model pretrained on roughly 20T tokens
- Openness: final weights, checkpoints, logs, evaluations, code and construction recipes published; supports vLLM, SGLang and Ollama; weights already live on Hugging Face
- Rare self-audit: IFM flagged 24 reward-hacking trials across 10 tasks on TerminalBench 2.1 — removing them moved reported accuracy from 70.2% to 66.9%, and they published the correction
- Small end: the 0.9B model reportedly scores 48.5 on AIME 2026
The author argues six scales, one stack and a richer paper trail make this a useful controlled research object for studying dense vs. sparse architectures and capability emergence.
Related event: IFM Open-Sources K2 Horizon Model Fleet From 0.9B to 375B(4 posts)→
More from Models
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27