105 real bugs tested: Muse Spark 1.3 ties Fable 5.1 at 33 as Meta joins the frontier
PawelHuryn · x · 2026-09-13
- PawelHuryn ran a private benchmark: 105 planted bugs across two real repos, original harness and API. Scores: Muse Spark 1.3 (max) 33, Fable 5.1 (high) 33, Grok 4.6 (xhigh) 27, Opus 5 (max) 27, Muse Spark 1.3 (high) 19. Verdict: "Meta joined the frontier."
- Methodology notes: bugs are real issues frontier models genuinely struggled with in early 2026; blind cross-family model judges, calibration confirmed; unplanted bugs don't count; answer key stays private, only anonymized logs published.
- Quirk: OpenAI models "bugmaxx"—reporting many real but irrelevant issues, and fixing them often complicates the solution.
- More effort levels dropping in the thread; Muse Spark 1.3 was the most requested model from the previous post.
Related event: Bug Hunt Bench: Muse Spark 1.3 Ties Fable 5.1 for the Top Spot(6 posts)→
More from coding & agent
- GitHub Copilot app is building richer dashboard widgets for personalized data views — mariorod1 · 2026-10-31
- Todoist's self-healing agent auto-fixes 1 in 5 Sentry issues and opens PRs — zeeg · 2026-10-31
- Dev's Multica 1.0 build log: 432M tokens, 89 agent runs, just $183 — jiayuan_jy · 2026-10-31
- Stardock's QA now runs agents that play the game and file reports — draginol · 2026-10-31
- Limma MCP structures unstructured data end-to-end so agents never miss a detail — FlimsyIngenuity7248 · 2026-10-31
- Agent audit metric: can you name which tests ran and files changed without reading its summary? — ronbodkin · 2026-10-31