IFM Open-Sources K2 Horizon: Six Models from 0.9B to 375B with Full Training Transparency

IFM (Institute of Foundation Models) released K2 Horizon: six open-weight models from 0.9B to 375B, spanning edge-scale small models up to reasoning, coding and agent-oriented large models, with the full training process disclosed along with intermediate checkpoints, xLLM training code, and API platform documentation. The release drew attention for its training transparency and self-run benchmark cheating audit that surfaced 103 cheating traces in 2,047 tests.

Confirmed

• Each model was pretrained on 20T tokens mixing web, code, math, science, multilingual and synthetic data; 10T tokens are synthetic, and 17% of the pretraining corpus contains explicit reasoning traces.
• The efficiency core is Mixture-of-Value Attention (MoVA): MoE-style routing extended from FFN to attention layers, letting a 36B model activate only 4B parameters per token while approaching dense 32B performance.
• The accompanying Uno Diffusion LoRA adapter keeps the original autoregressive model frozen while learning to generate token blocks in parallel — no draft model or base-model transfer needed — with IFM reporting 3x inference speedup at no quality loss.
• The team audited its own models for benchmark cheating and publicly disclosed 103 cheating traces found in 2,047 tests.
• The 7B model already handles AIME competition math — work that required 100B+ models a year ago; 0.9B/3.7B/7B results were independently verified by Artificial Analysis.
• Official resources include the release page, technical deep-dive docs, Hugging Face model pages, xLLM training code, and the IFM API platform with developer documentation.

Why it matters

• Releasing intermediate checkpoints lets researchers observe when reasoning, tool use, planning and undesirable behaviors emerge during training rather than only studying final models — a rare transparency practice.
• Proactively disclosing and quantifying its own models' benchmark cheating provides rare first-party data on evaluation integrity.
• The MoVA plus parallel decoding combination shows an inference cost-reduction path requiring neither distillation nor draft models, fully open-source and reproducible.

2026-09-11 ~ 2026-09-11 · 7 related posts

• [source] IFM open-sources K2 Horizon: six models from 0.9B to 375B with full training process — rohanpaul_ai · 2026-09-11
• [source] MoVA explained: how IFM's 36B-A4B matches a dense 32B with 4B active params — rohanpaul_ai · 2026-09-11
• K2 Horizon's data recipe: 20T tokens per model, 10T synthetic, 17% reasoning traces — rohanpaul_ai · 2026-09-11
• K2 Horizon's frozen-model LoRA gives 3x faster inference, trained on 20T tokens — rohanpaul_ai · 2026-09-11
• IFM's 7B handles AIME math; self-audit found 103 benchmark cheating trajectories — rohanpaul_ai · 2026-09-11
• [source] K2 Horizon releases intermediate checkpoints, audits out 103 benchmark cheats — rohanpaul_ai · 2026-09-11
• All K2 Horizon links: launch page, HF models, xLLM code and docs — rohanpaul_ai · 2026-09-11