Bug Hunt Bench: Muse Spark 1.3 Ties Fable 5.1 for the Top Spot

Developer Pawel Huryn built and maintains Bug Hunt Bench: 105 real bugs planted in two production-grade codebases, with frontier coding models tested agentic-style via raw APIs to find and fix them. On September 13 he tested Meta's new Muse Spark 1.3 with this harness; its max tier tied Fable 5.1 (high) at 33 points for first place, leading him to argue Meta has entered the frontier coding tier.

Confirmed

• Setup: two real repos, 105 planted bugs, covering GPT-6, Claude, Grok, Gemini, DeepSeek, Kimi, GLM and other frontier models
• Latest ranking: Muse Spark 1.3 (max) and Fable 5.1 (high) tied at 33 points; Grok 4.6 (xhigh) close behind
• Tier testing: Muse Spark 1.3's xhigh tier scored 20—50% more expensive and 13% slower than high, solving only one extra bug; the author sees high→max as the biggest jump
• Medium tier ranked 9/105; the author recommends using high or higher reasoning tiers in practice

Why it matters

• The benchmark uses real-repo bugs rather than synthetic tasks, closely reflecting real-world debugging, offering differentiated guidance for model selection
• Muse Spark 1.3 tying for first suggests Meta has reached or is approaching the frontier tier in coding
• Tier cost-benefit data (low marginal gain for xhigh, high gain for high→max) directly informs inference-cost control

2026-09-13 ~ 2026-09-14 · 6 related posts

• [source] Muse Spark 1.3 ties Fable 5.1 in 105-bug repo test, Meta joins frontier — PawelHuryn · 2026-09-13
• 105 real bugs tested: Muse Spark 1.3 ties Fable 5.1 at 33 as Meta joins the frontier — PawelHuryn · 2026-09-13
• [source] Muse Spark 1.3 Tested: xhigh Costs 50% More, 13% Slower, Fixes One Extra Bug — PawelHuryn · 2026-09-13
• [source] Bug Hunt Bench grades frontier coding models on 105 real bugs in production repos — PawelHuryn · 2026-09-13
• Muse Spark 1.3 (medium) lands at 9/105 on benchmark; use high+ instead — PawelHuryn · 2026-09-13

1 near-duplicate retellings: PawelHuryn