K2 Horizon's data recipe: 20T tokens per model, 10T synthetic, 17% reasoning traces
rohanpaul_ai · x · 2026-09-11
Details of K2 Horizon's training data:
- Each model pretrained on 20T tokens mixing web, code, math, science, multilingual and synthetic data
- 17% of the pre-training corpus contains explicit reasoning trajectories; 10T synthetic tokens were used in pre-training
- Post-training used 100M+ unique tasks, with the pipeline released via SFT, model merging, RL and specialized agent training
- The released pre-training dataset TxT360-v2 (5.3 TB) is already on Hugging Face
More from Models
- Tencent releases Hy-SimulMT: real-time speech-to-speech translation with ~3s latency — _akhaliq · 2026-10-28
- Meta's Omnilingual MT covers 1,600 languages, 1B–8B models rival 70B LLMs — kalyan_kpl · 2026-10-28
- Uncensored LFM3-1.7B GGUF weights released on Hugging Face — kalyan_kpl · 2026-10-28
- Token demand on OpenRouter grew 27x to 165T weekly, open models now serve 72% — FinanceYF5 · 2026-10-28
- Tesla Optimus 'robot university' leak: 40-camera mocap suits, 24/7 data ops for months — Scobleizer · 2026-10-28
- Letta's FrontierMemory leaderboard: Letta Dream tops at 73.3%, beating OpenAI and Claude agents — afurgs · 2026-10-28