Meta's Muse Spark 1.3 underwhelms on cybersecurity benchmark: 19/32 CVEs vs Grok 4.6's 23.3
teortaxesTex · x · 2026-09-03
Third-party testers ran Meta's newly launched Muse Spark 1.3 through their cybersecurity benchmark:
- At pass@1, the model rediscovered an average of 19/32 CVEs, trailing Grok 4.6's 23.3/32
- Pooling three runs (pass@3) yielded 24/32, still behind DeepSeek V4 Pro at 28/32
- Pricing is competitive with frontier models but hard to justify versus some Chinese models
- Cached input token rate was just 45%, lower than usual; results were also normalized assuming 90%
Verdict: Meta is slowly catching up — decent at coding, not yet there for security. Zuckerberg framed the launch as Meta's biggest coding/agentic jump, with open weights and a mysterious 🍉 release teased.
More from Models
- Mistral's Voxtral 3 speech models land #3 on STT leaderboard at $13/1M chars — ArtificialAnlys · 2026-10-21
- AI2's FairStress benchmark exposes hidden fairness asymmetries across 11 open-weight models — allen_ai · 2026-10-21
- Perplexity ships Decider V1.2, tops Vals AI's new ClaimBench decision-model benchmark — bo_wangbo · 2026-10-21
- Omnii-1 genome model debuts: first AI-designed phages ever given to a human patient — JosephJacks_ · 2026-10-21
- Nous Portal Pricing Revealed: Unused Credits Roll Over, Up to $100 Cap on $200 Tier — alexcovo_eth · 2026-10-21
- 1.3B open-source doc parser OXR tops three benchmarks, parsing 100 pages in 101 seconds — s_mohinii · 2026-10-21