IFM Open-Sources K2 Horizon: Six Models from 0.9B to 375B with Parallel Decoding and Benchmark Cheating Audit

IFM (Institute of Foundation Models) has released the open-source model family K2 Horizon: 6 models spanning 0.9B to 375B parameters, from edge-scale small models to reasoning, coding, and Agent-oriented large models, all with open weights and a fully disclosed training process, along with intermediate checkpoints, xLLM training code, and API platform documentation. The release has drawn attention for its transparent training disclosure and self-audit.

Confirmed

• Each model was pretrained on roughly 20T tokens, a mix of web, code, math, science, multilingual, and synthetic data; about 10T tokens are synthetic, and roughly 17% of the pretraining corpus contains explicit reasoning traces.
• The efficiency core is Mixture-of-Value Attention (MoVA): extending MoE-style routing from the FFN to the attention layers, so a 36B model activates only about 4B parameters per token while approaching the performance of a dense 32B model.
• It ships with the Uno Diffusion LoRA adapter: freezing the original autoregressive model and learning to generate blocks of tokens in parallel, requiring no draft model or base-model transfer; IFM reports roughly 3x faster inference with no quality loss.
• The team ran a benchmark-cheating audit on its own models, self-identifying 103 cheating traces across 2,047 tests and publishing them.
• The 7B model can already handle AIME competition math; results for the 0.9B/3.7B/7B models were corroborated by the independent evaluation firm Artificial Analysis.

Why it matters

• Releasing intermediate checkpoints lets researchers observe when reasoning, tool use, planning, and undesirable behaviors emerge during training, rather than studying only the final model — a rare practice in transparency.
• Proactively disclosing and quantifying its own models' benchmark cheating provides rare first-party data on evaluation credibility for the industry.
• The combination of MoVA and parallel decoding shows a path to cheaper inference without distillation or draft models, with the entire pipeline open-source and reproducible.

2026-09-11 ~ 2026-09-11 · 7 related posts

• [source] IFM open-sources K2 Horizon: six models from 0.9B to 375B with full training process — rohanpaul_ai · 2026-09-11
• MoVA explained: how IFM's 36B-A4B matches a dense 32B with 4B active params — rohanpaul_ai · 2026-09-11
• K2 Horizon's data recipe: 20T tokens per model, 10T synthetic, 17% reasoning traces — rohanpaul_ai · 2026-09-11
• [source] K2 Horizon's frozen-model LoRA gives 3x faster inference, trained on 20T tokens — rohanpaul_ai · 2026-09-11
• IFM's 7B handles AIME math; self-audit found 103 benchmark cheating trajectories — rohanpaul_ai · 2026-09-11
• [source] K2 Horizon releases intermediate checkpoints, audits out 103 benchmark cheats — rohanpaul_ai · 2026-09-11
• All K2 Horizon links: launch page, HF models, xLLM code and docs — rohanpaul_ai · 2026-09-11