How Kimi K3 engineered its way to the frontier: Delta Attention, expert balancing, AgentENV
noninertialframe96 · reddit · 2026-07-31
Moonshot's Kimi K3 reached frontier level, ranked 4th of 580 models by Artificial Analysis (behind Claude Opus 5, Fable 5, GPT-5.6 Sol). Technical report highlights:
- Kimi Delta Attention: replaces KV cache in 69/93 layers with a 128x128 matrix per head; 1M-token context uses 27.2 GiB vs 104.6.
- Quantile Balancing: keeps 896 experts/layer evenly loaded by computing bias directly.
- AgentENV: Firecracker microVM runtime created 51M sandboxes with 133ms checkpoints and 49ms resumes.
Full walkthrough: https://codepointer.substack.com/p/how-kimi-k3-engineered-its-way-to
More from Models
- Trying Grok bot surfaces an unpaid Cursor balance, blocking account merging — haydendevs · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- Leaked screenshot shows unannounced Gemini 4 Argon gated to Ultra and paid Pro subscribers — il_-bro · 2026-10-16
- Google: Gemini 4 Argon lands in Antigravity, code submissions up 35%, rollbacks down ~33% — tekbog · 2026-10-16
- Google's "ungrounded beliefs" classifier injections derail Opus 4.1's reasoning chain — liminal_bardo · 2026-10-16
- A week of ChatGPT Pro quota burned in 36 hours as GPT-6 fails to factor a 512-bit RSA key — matthew_d_green · 2026-10-16