CyberBinGym scores: Codex 88% recovery / 20% exploit, Kimi Code 0% exploitation
dawnsongtweets · x · 2026-11-12
CyberBinGym thread results (vulnerability recovery / end-to-end exploitation): Codex (GPT-5.6 Sol) 88%/20%, Claude Code (Claude Opus 5) 85%/6%, Kimi Code (Kimi K3) 64%/0%.
Scoring is split: recovery requires a PoC that crashes only the vulnerable build under sanitizers; exploitation requires capturing a secret flag from a separate victim service. All three agents recover most vulnerabilities from patches, but turning them into working exploits remains far harder — though binary-only exploitation is now within reach.
More from Models
- Fable 5.2 cuts refusals dramatically but slides to 40th & 53rd on BullshitBench — burny_tech · 2026-11-12
- Artificial Analysis: Upgrading Claude model tier beats maxing reasoning effort on cost — ArtificialAnlys · 2026-11-12
- Google Showcases Gemini 4 Carbon: Building a Custom CUDA/C++ VFX Engine From Scratch — burny_tech · 2026-11-12
- Claude iOS spotted adding 'Saved logins' for signing into websites, fueling Duet speculation — testingcatalog · 2026-11-12
- Fable 5.2 reverse-engineers how 1400s composers wrote four-voice counterpoint, then composes — imjustnewatai · 2026-11-12
- GPT-Live talk at VapiCon: OpenAI pushes natural speech with frontier intelligence — juberti · 2026-11-12