Pawel Huryn Benchmarks Muse Wonder 1.0: GPT-6 Astra Leads at 45 Points

AI researcher Pawel Huryn built a real-world benchmark: two real code repositories and 105 bugs that no Q1 2026 frontier model had found, with each model run at max setting to try to find and fix them, measuring real-world engineering bug-fixing ability. GPT-6 Astra scored 45.0 to take first place, Opus 5.5 got 41.7, while Muse Wonder 1.0 fixed only 31.7 bugs at a 50% higher price. The results show that on problems close to real-world conditions, gaps between models remain large, and Muse Wonder's price-performance is poor.

Confirmed

• Benchmark setup: two real codebases, 105 bugs no Q1 2026 frontier model had found, run at max setting
• Scores: GPT-6 Astra (max) 45.0, Opus 5.5 (max) 41.7; Muse Wonder 1.0 fixed 31.7 bugs
• Muse Wonder 1.0 costs 50% more than its rivals
• Across all benchmark runs, 29 bugs have still never been fixed by any model

Why it matters

• Huryn notes that new models can indeed fix some bugs in real environments, but often only after he or other users first encounter them in the app — meaning proactive discovery remains a weak spot
• He endorses filtering out easy tasks and keeping only problems 'old models couldn't crack,' since simple bugs push all model scores toward convergence and destroy discriminative power
• The 29 long-unsolved tail bugs in SWE-bench could serve as a key sample for measuring the real progress of the next wave of models

2026-10-13 ~ 2026-10-13 · 5 related posts

• [source] Real-world test: Muse Wonder 1.0 fixes 31.7/105 bugs, 50% pricier than Spark 1.3 — PawelHuryn · 2026-10-13
• [source] Real-world test: GPT-6 Astra tops bug-hunt benchmark with 45.0, finds 105 bugs frontier models missed — PawelHuryn · 2026-10-13
• [source] 29 bugs in benchmark history have never been fixed by any model, says Pawel Huryn — PawelHuryn · 2026-10-13
• 29 Bugs in Benchmark History Have Never Been Fixed by Any Model — PawelHuryn · 2026-10-13

1 near-duplicate retellings: curious_mahesh