SEC-bench Pro: Gemini 4 Carbon leads at 88.2% Pass@1, DeepSeek models trail at 56-63%
ColbyHawker · x · 2026-10-29
ColbyHawker jokes that Gemini 4 Carbon is "really good at looking for flaws, like your neighborhood HOA," sharing Google DeepMind's SEC-bench Pro (Pass@1) results:
- Gemini 4 Carbon: 88.2%
- GPT-6 Astra: 85.4%, Gemini 4 Argon: 83.4%, GPT-6.1 Terra: 83.3%, GPT-6.1 Sol: 78.8%
- MiMo-V2.6-Pro: 66.3%, DeepSeek-V4.1-Flash: 62.8%, DeepSeek-V4-Pro: 56.4%
Gemini 4 Carbon clearly leads on this security-review benchmark, with DeepSeek models at the bottom of the chart.
Related event: Google launches Gemini 4 Carbon, topping the Artificial Analysis index(84 posts)→
More from Models
- New benchmarks suggest Kimi K3.1 improves on hard tasks despite skipping SWE-maxxing — teortaxesTex · 2026-10-29
- Insider claims Anthropic is sandbagging a bigger lab announcement due Q4-end or Q1 — ChrisGPT · 2026-10-29
- Information-theoretic scaling laws predict memorization and FID in diffusion models — LucaAmb · 2026-10-29
- Early Hands-On: Google's Carbon Beats Argon With Less Slop, Better Constraint Adherence — legit_api · 2026-10-29
- LMArena tests Gemini 4 Carbon preview: strong on 3D worlds and creativity, gaps in detail — arena · 2026-10-29
- Gemini 3.1 Pro split-persona breakdown: two selves fight over the browser — davidad · 2026-10-29