Kimi K3’s cyber eval still looks weak, but it can finish a full long-horizon chain
dyn___ · x · 2026-07-24
A repost of an AI security evaluation argues that the Kimi K3 cyber score should be read carefully, but the underlying signal still matters.
- The critic says the published “cyber capability” label is a stretch because it is essentially backed by ExploitBench.
- At the same time, they note that Kimi K3’s 1/10 full TLO solve is still meaningful: reliability is low, but the model can complete the whole long-horizon chain in this setup.
- The bigger point is that benchmark rankings are not the same as real-world capability; with better scaffolding and expert guidance, performance can look different.
- The chart in the attached image compares Kimi K3 and other PRC models against top U.S. models on overall cyber capability and ladder scores.
Related event: Kimi K3 Cybersecurity Eval Sparks Debate: Scores 32.2%(5 posts)→
More from Models
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- DeepSeek V4 Pro API to continue after Sept 2026, billing unchanged — teortaxesTex · 2026-09-11
- DeepSeek V4.1 Flash Hits 98% of GPT-6 Astra's Score at 1.4% of the Cost in Third-Party Benchmark — ayushtweetshere · 2026-09-11
- TheZvi Polls: Has Your Coding Model Choice Changed Since Fable 5.1 and Astra? — TheZvi · 2026-09-11
- antirez Weighs In on Anthropic Banning Minors From Using Claude — antirez · 2026-09-11