Moonshot deliberately curbed Kimi K3.1's cyber capabilities: 74.7 on ExploitBench vs GPT-6 Astra's 100
bookwormengr · x · 2026-10-24
- Kimi K3.1 Pro scores 74.7 on ExploitBench and 20.7 on ExploitGym, well below GPT-6 Astra (100 / 42.7); Claude Fable 5.1 gets 83.0/30.4 and Claude Opus 5.5 91.0/34.5.
- According to the poster, Moonshot intentionally reduced cyber capability in the open-weights model by filtering training datasets, since these capabilities were 'emerging' on their own rather than trained via RL on exploit tasks.
- Among open models, DeepSeek-V4.1-Flash has no reported ExploitBench/Gym scores (85.7 on internal xdev-bench-hard), while GLM-5.3 scores 54.4 and 15.0.
Related event: Moonshot admits deliberately weakening Kimi K3.1 Pro's cyber capabilities(8 posts)→
More from Models
- Dev notes model fixed overthinking with fewer reasoning tokens, suspects benchmaxxing — cedric_chee · 2026-10-24
- Public benchmarks show Kimi K3 beating all OpenAI models, including Astra — bindureddy · 2026-10-24
- First Kimi K3.1 Pro benchmarks compare it against Opus 5.5, Sonnet 5.5 and Gemini 4 — arena · 2026-10-24
- Recursive's 40B Flint Hits ~200 tok/s on M3/M5 Ultra via MLX-VLM with Fused Metal Kernels — timshi_ai · 2026-10-24
- Richard Socher launches Flint, an open-weights local coding model hitting 125+ tokens/s on a MacBook — RichardSocher · 2026-10-24
- Anthropic's unannounced "Claude Duet" app spotted testing on iOS at version 1.999999 — kevinakwok · 2026-10-24