Kimi K3: Open Frontier Intelligence
Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, M. C., Jianfeng Cai, Xinyuan Cai, Peizhou Cao, Yuxuan Cao, Ziwei Chai, Y. Charles, H. S. Che, Guanduo Chen, Guangyu Chen, Guanzheng Chen, Huarong Chen, Jia Chen, Jianlong Chen, Jun Chen, Kexin Chen, Peng Chen, Ruijue Chen, Wentao Chen, Xin Chen, Yang Chen, Yanru Chen, Yifei Chen, Yingjiang Chen, Yuankun Chen, Yujie Chen, Yutian Chen, Zhirong Chen, Dazhi Cheng, Yean Cheng, Jialei Cui, Jingbing Cui, Anqi Dai, Jiaqi Deng, Hao Ding, Rui Ding, Shaofeng Ding, Mengfan Dong, Mengnan Dong, Yuhao Dong, Yuxin Dong, Angang Du, Chenzhuang Du, Dikang Du, Jusen Du, Yulun Du, Yu Fan, Jing Feng, Qiulin Feng, Yichen Feng, Kelin Fu, Qiang Fu, Fuxuan Gao, Hongcheng Gao, Jingyue Gao, Tong Gao, Weijia Gao, Shangyi Geng, Jie Gong, Linhu Gong, Shengao Gong, Xiaochen Gong, Qizheng Gu, Yicheng Gu, Shuhao Guan, Haiqing Guo, Shiqi Guo, Xiang Guo, Zhengyan Guo, Beixi Hao, Wenxin Hao, Xiaoru Hao, Dailan He, Haotian He, Lehan He, Qi He, Weiran He, Xinran He, Xinyi He, Yibo He, Yunjia He, Chao Hong, Tiange Hong, Hao Hu, Jiaxi Hu, Ruikun Hu, Weiming Hu, Yangyang Hu, Zhenxing Hu, Liang Hua, Jinbin Huang, Ke Huang, Ruiyuan Huang, Siying Huang, Weixiao Huang, Yan Huang, Zhengjie Huang, Zhiqi Huang, Yulong Hui, Chaobo Jia, Yutong Jiang, Zhejun Jiang, Zuoyou Jiang, Wenyi Jin, Xinyi Jin, Yu Jing, Huanjun Kong, Guokun Lai, Aidi Li, Cheng Li, Chengyuan Li, Cong Li, Fang Li, Guanyu Li, Haoyang Li, Jia Li, Junxiong Li, Lei Li, Letian Li, Lincan Li, Weihong Li, Wentao Li, Xintong Li, Yang Li, Yishen Li, Yiwei Li, Yuxiao Li, Zhaowei Li, Zhaoxi Li, Zheming Li, Zhengxiao Li, Zhiyuan Li, Jiawei Lin, Xiaohan Lin, Yibo Lin, Zichao Lin, Ziyan Lin, Bill Liu, Boxiao Liu, Chuan Liu, Liang Liu, Shaowei Liu, Shudong Liu, Shuran Liu, Tianwei Liu, Weizhou Liu, Yangyang Liu, Yanming Liu, Yibo Liu, Yipeng Liu, Zhengying Liu, Zhiheng Liu, Enzhe Lu, Haoyu Lu, Linqiang Lu, Tingzhan Lu, Zhiyuan Lu, Aotian Luo, G. Luo, Junyu Luo, Yifan Luo, B. Lyu, Wenzhou Lyu, Shaoguang Mao, Yuan Mei, Xin Men, Minqing Ni, Yixuan Niu, Siyuan Pan, Shujun Peng, Zhangyang Qi, Ruoyu Qin, ZeChao Qin, Zeyu Qin, Haiquan Qiu, Jianxin Qiu, Jiezhong Qiu, Bowen Qu, Yuhao Qu, Zeyu Shang, Youbo Shao, Han Shen, Jincheng Shi, Juanfeng Shi, Lidong Shi, Shengyuan Shi, Wingchun Siu, Pengwei Song, Xiaoxi Song, Jianlin Su, Yunfeng Su, Zhaochen Su, Lin Sui, Jingsong Sun, Junyao Sun, Shaoning Sun, Shuzhe Sun, Tongyu Sun, Yujun Sun, Yunpeng Tai, Chuning Tang, Heyi Tang, Sirui Tang, Zecheng Tang, Chaoran Tian, Rongpeng Tian, Yu Tian, Wei Tu, Chensi Wang, Chuang Wang, Chunjie Wang, Dinglu Wang, Feng Wang, Hailong Wang, Haiming Wang, Hao Wang, Hao Wang, Huaqing Wang, Hui Wang, Jiayi Wang, Jinglong Wang, Jinhong Wang, Jiuzheng Wang, Linian Wang, Shaobo Wang, Shenzhi Wang, Shuyi Wang, Si Wang, Siyuan Wang, Tianfu Wang, Wenjue Wang, Xingran Wang, Xinmei Wang, Xinyuan Wang, Xusheng Wang, Yalin Wang, Yangkun Wang, Yao Wang, Yaoyu Wang, Yejie Wang, Yiqin Wang, Yucheng Wang, Yuzhi Wang, Zhaoji Wang, Zhaowei Wang, Zhengtao Wang, Zhenhao Wang, Zhongsheng Wang, Zifan Wang, Chu Wei, Ming Wei, Shouxin Wei, Zichen Wen, Fan Wu, Haoning Wu, Rucong Wu, Wenhao Wu, Xiaoxue Wu, Yingcong Wu, Yongqi Wu, Yuxin Wu, Zijian Wu, Xinglang Xian, Chenxuan Xiang, Yuye Xiang, Bocheng Xiao, Chenjun Xiao, Xin Xiao, Jin Xie, Xiaotong Xie, Yifeng Xie, Zhe Xie, Bowei Xing, Yiming Xiong, Baosheng Xu, Boyu Xu, Jiale Xu, Jianfan Xu, Jing Xu, Jinjing Xu, L. H. Xu, Qingtao Xu, Shuyao Xu, Suting Xu, Tiantian Xu, Tianxiang Xu, Weixin Xu, Xinran Xu, Yangchuan Xu, Ye Xu, Yueni Xu, Ziyao Xu, Haonan Xue, Junjie Yan, Yaoyao Yan, Fan Yang, Guangyao Yang, Hao Yang, Junwei Yang, Ruoyu Yang, Wenjie Yang, Xiaofei Yang, Xinyu Yang, Yi Yang, Yiling Yang, Ying Yang, Yuchen Yang, Zhen Yang, Zhilin Yang, Zian Yang, Zuhao Yang, Haotian Yao, Dan Ye, Haoran Ye, Wenjie Ye, Zhanbo Ye, Bohong Yin, Haoxiang Yin, Xietong Yin, Chengzhen Yu, Haozhen Yu, Longhui Yu, Shengnan Yu, Shuying Yu, Tianxiang Yu, Enming Yuan, Mengjie Yuan, Tongtian Yue, Wei Yue, Yang Yue, Dunyuan Zha, Haobing Zhan, B. H. Zhang, Dehao Zhang, Fei Zhang, Hao Zhang, Haoyuan Zhang, Huanyu Zhang, Jiapei Zhang, Jiaxuan Zhang, Jin Zhang, Kaiyi Zhang, Miaozhen Zhang, Puqi Zhang, Qinglei Zhang, Rong Zhang, Rui Zhang, Shaoshuai Zhang, Shiyi Zhang, Xiaobin Zhang, Xiaoyun Zhang, Y. Zhang, Yangkun Zhang, Ye Zhang, Yichi Zhang, Yikun Zhang, Yizhi Zhang, Yongting Zhang, Yu Zhang, Yutao Zhang, Yutong Zhang, Zheng Zhang, Zijing Zhang, Bin Zhao, Chenguang Zhao, Feifan Zhao, Jinglun Zhao, Jinxiang Zhao, Shuai Zhao, Wenshuo Zhao, Xiangyu Zhao, Xuanle Zhao, Yikai Zhao, Zijia Zhao, Haozhi Zheng, Huabin Zheng, Ruihan Zheng, Shaojie Zheng, Tengyang Zheng, Haofeng Zhong, Lei Zhong, Longguang Zhong, M. Zhou, Qiankang Zhou, Runjie Zhou, Ruozhang Zhou, Xinyu Zhou, Yiqiao Zhou, Zaida Zhou, Jinguo Zhu, Liya Zhu, Xinhao Zhu, Yangjunfeng Zhu, Yuxuan Zhu, Zhen Zhu, Chen Zhuang, Weiyu Zhuang, Xinxing Zu
cs.CL, cs.LG
2026-07-28
Kimi 团队放出 2.8T 参数、104B 激活、1M 上下文的开源 MoE,scaling 效率比 K2 高约 2.5 倍;40+ 评测紧追 Claude Fable 5 与 GPT-5.6 Sol,推理成本只有旗舰零头。
开源大模型这两年在「测试时算力」(test-time scaling)这条线上进步飞快,推理、强化学习、长程 agent 都有突破。但在另一条更基础的线「预训练规模」上几乎停滞,主流模型卡在 1T 级参数附近。结果是推理方法越卷越花,底座却没怎么长,和最强闭源模型的差距反而在拉大。Kimi K3 想同时把两条线推到前沿:把预训练底座做到史无前例的 3T 级,同时在 1M 上下文里把 RL、推理努力度和长程交互也拉满。
K3 是原生多模态 MoE,总参数 2.8T,每 token 激活 104B,上下文 1M。架构沿三个维度扩信息流。
序列维度用混合注意力:每个 block 是三层 Kimi Delta Attention(KDA)夹一层 Gated MLA 保留全局交互。深度维度用 Attention Residuals,每层可以越过常规的顺序残差,选择性回看前面任意层和 embedding 的表示。宽度维度用 Stable LatentMoE,路由专家扩到 896 个,每 token 激活 16 个,极端稀疏下靠归一化、SiTU-GLU 激活和 Quantile Balancing 稳住优化。视觉编码器 MoonViT-V2 从头用 next-token prediction 训练,401M 参数。
相比 K2(1.04T 总参、32.6B 激活、128K 上下文、MLA 注意力),K3 在层数(61→93)、激活参数(32.6B→104.2B)、专家数(384→896)、上下文(128K→1M)上都大幅放大。论文用 scaling law 研究把这些改动连同数据、训练配方的改进算在一起,在外推验证集上相对 K2 有约 2.5 倍的整体 scaling 效率提升,即达到同等 validation loss 所需算力降到约四成。
后训练面向 1M 上下文的测试时扩展:在长程编码、通用 agent、推理、知识这几个域、多个推理努力度上做 RL,训练环境覆盖可验证搜索、软件工程、kernel 优化、视觉工具调用、持久化助手工作流、Web 开发、自主执行,一个回合里往往几百上千次工具调用。各域各努力度的策略再通过多教师在线策略蒸馏合并成一个模型。
40 多个 benchmark,跟 Claude Fable 5、GPT-5.6 Sol、Claude Opus 4.8、GPT-5.5、GLM-5.2 比(K3 用 max 推理努力)。整体定位:紧追 Fable 5 和 GPT-5.6 Sol,稳定领先其他几个。挑几个有代表性的:
| benchmark | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol |
| GPQA Diamond | 93.5 | 92.6 | 94.1 |
| SWE-Marathon(GPU kernel) | 42.0 | 35.0 | 39.0 |
| FrontierSWE | 81.2 | 86.6 | 71.3 |
| BrowseComp | 91.2 | 88.0 | 90.4 |
| MCPMark-Verified | 94.5 | 87.4 | 92.9 |
研究级推理是短板:CritPt 只有 23.4%(Fable 5 28.6、GPT-5.6 32.3),HLE-Full 带工具 56.0%(Fable 5 63.0)。第三方榜单上,Intelligence Index v4.1 排第 4/580(57.1),Vals Index 第 2/39(74.7),WebDev Arena Elo 第 1/99(1678)。成本侧亮点明显:BrowseComp 91.2 分、单任务 2.03 美元,是 GPT-5.6 Sol(90.4 分)的一半左右,比 Claude 旗舰便宜一个数量级。
这是第一个公开权重的 3T 级模型。开源圈长期缺一个能和闭源旗舰在底座规模上正面打的选手,K3 把参数、激活量、上下文都顶上去之后,在编码、agent、知识、视觉上一大批任务做到接近闭源旗舰、远超其他开源模型,推理成本却只有旗舰的零头。对自托管、做 agent 产品的团队,这是一个可拿来做底座或对照的真选项。权重完全开放。
作者自己说,研究级推理(CritPt、HLE)落后 Fable 5 和 GPT-5.6 Sol 一截,是下一步重点;几个 Elo 评级的知识工作套件(GDPval-AA v2、AA-Briefcase)被 Fable 5 压一头。横向比要打折扣:对比里 Fable 5 的结果含 fallback、GPT-5.6 Sol 含 cyberguard,各模型用的 harness(Kimi Code / Claude Code / Codex)也不同,结果对 harness 敏感,并不完全干净。Elo 榜单分数会随对局累积漂移。要跑 2.8T、1M 上下文,部署门槛很高,论文 infra 章节占了大篇幅正说明这点。