Kimi K3 paper details SFT data synthesis, XTLM chat templates, and QAT
stochasticchasm · x · 2026-07-28
The paper excerpt describes the supervised fine-tuning stage for Kimi K3. It says the team expanded the SFT dataset by synthesizing trajectories from earlier Kimi models, then verified and annotated them with human-in-the-loop review.
A notable detail is that all data are serialized using an XTLM-based chat template (Extensible Token Markup Language) to keep complex agentic trajectories consistent. The excerpt also says quantization-aware training (QAT) is applied from the SFT stage onward, using MXFP4 weights and MXFP8 activations to support training and deployment efficiency.
Related event: Kimi K3 Technical Report Details 3-Stage Post-Training(4 posts)→
More from Models
- k3 adds no RL algorithm changes, while tool-call steps track eval scores almost 1:1 — stochasticchasm · 2026-07-28
- DeepInfra adds Claude Opus 5 with 1M context and $5/$25 pricing — gharik · 2026-07-28
- Kimi K3 goes live on OpenRouter as third-party providers race to add support — scaling01 · 2026-07-28
- Kimi K3 arrives on Applied Compute for training and inference — rhythmrg · 2026-07-28
- A repost claims Anthropic’s Opus 5 regresses badly despite benchmark gains — rickasaurus · 2026-07-28
- Polymarket prices a 76% chance Moonshot ships another Kimi K model by September — Polymarket · 2026-07-28