Kimi K3 beats GLM-5.2 in exploit tests but still fails end-to-end attacks
kevinsxu · x · 2026-07-28
The post quotes the Kimi K3 paper’s cyber evaluation results and notes that a joint UK AI Security Institute / NIST CAISI assessment reached similar conclusions.
- Kimi K3 outperforms GLM-5.2 on exploit development.
- It scores 32% vs. 24% on ExploitBench.
- On a simulated enterprise network task that takes a human expert about 20 hours, it needs 17 steps vs. 11 steps.
- But it still trails frontier cyber-capable models on end-to-end exploit completion, achieving arbitrary code execution on 0 of 41 tasks.
The punchline is that China and the US institutions agree on the ranking, at least on this slice of cyber capability.
Related event: Kimi K3 Security Eval: High Exploit, Low Guardrails(4 posts)→
More from Models
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27