K2 Horizon tech report: 134 pages on building six frontier-scale LLMs
HongyiWang10 · x · 2026-11-18
K2 (LLM360/IFM) released the K2 Horizon technical report on arXiv — 134 pages covering how six models were built, from architecture to inference, billed as the most detailed LLM training reference available today. Key points: two new attention designs (MoVA and softplus attention gating) let a 36B model nearly match a 32B dense model with 1/7 the active parameters; four models from 3.7B to 36B trained on the same 22T tokens show nearly identical normalized loss curves, meaning small-model recipes transfer at scale. Eric Xing highlights full transparency and reproducibility as something corporate LLM producers don't offer.
Related event: IFM Releases K2-Horizon: Six Open Models Plus a 134-Page Technical Report(7 posts)→
More from Models
- Dev's Deep Dive: GPT-6.2 Astra Improved but Still 'Directionally Very Bad' at Debugging — kimmonismus · 2026-11-18
- OpenAI Charged for a ChatGPT Business Premium Seat for a Month That Can't Be Used — physmathastro · 2026-11-18
- Hunch: Astra 6.2's bad reviews stem from OpenAI dogfooding with unlimited internal swarms — daniel_mac8 · 2026-11-18
- Google rolls out Antigravity Orchestrator: dynamic in-task model switching, 24% less quota — rseroter · 2026-11-18
- Qwen3.8-27B OpenCode adapter lifts SWE-Bench Pro 35.3%→51% with 6-8x fewer tokens — yb2698 · 2026-11-18
- HF's Merve Noyan: Liquid AI is killing it with tiny VLM, Omni and decision models — helloiamleonie · 2026-11-18