Mistral Large 4 Beats Opus 5.5 and GPT-6 Astra on Cybersecurity Benchmarks — Lower Refusal Rates
burny_tech · x · 2026-10-07
Per @cline, Mistral Large 4 outperforms Claude Opus 5.5 and GPT-6 Astra on cybersecurity benchmarks, largely because it refuses far fewer security tasks — while Opus and Astra had roughly 40% of tasks blocked by their own safety filters. A notable data point on how stricter alignment can hurt scores on security-adjacent benchmarks, from a team that runs these models heavily in production.
More from Models
- Qwen3.8 Flash Next GGUF benchmark: IQ3_S the sweet spot, 42.7M tokens tested — lxfater · 2026-10-07
- Developer regrets using Haiku to save tokens on a 'simple' task — geoffwolfe · 2026-10-07
- Grok Bot Impresses but Lacks Local Computer Use, Sticking to Cloud Machines — mark_k · 2026-10-07
- Developer finds xAI's Grok bot impressively useful for SEO, unlike Grok 4.5/4.7 — gaganghotra_ · 2026-10-07
- Musk says SpaceX's Grok bot will route tasks to Claude Opus 5.5, MidJourney, Suno and other top APIs — elonmusk · 2026-10-07
- OpenAI enables textGrain text watermarks for EU users with no opt-out — Banania2020 · 2026-10-07