Muse touts new coding model on ProgramBench, where top models barely hit 4.5%
jyangballin · x · 2026-10-13
Muse team member jyangballin announced Muse Wonder's results on ProgramBench, calling the model "very good at coding" and teasing that % resolved numbers on the leaderboard will "blow your mind" later this week.
What ProgramBench is: a new benchmark from Meta Superintelligence Labs, Stanford, and Harvard where agents must rebuild a complete codebase from scratch given only a compiled binary and its docs — 200 tasks, judged by hidden behavioral tests.
Leaderboard today: Claude Opus 5 (xhigh) leads with 4.5% fully resolved and 37.0% almost resolved at a hefty $50.53 per task; Muse Spark 1.3 (max) scores 2.5%/25.0% at just $6.46; GPT-5.6 Sol, GPT 5.5, and Gemini 3.6 Flash all sit near or below 1% fully resolved.
Per the attached chart: Muse Wonder Full leads on DeepSWE 1.1 (79.0) and SWE-Marathon (57.5) against Claude Fable 5.1 and GPT-6 Astra, but trails Claude Opus 5.5 on ProgramBench (59.0 vs 70.5) and the Artificial Analysis Index.
Key signal: even frontier models fully solve only single-digit percentages of these ultra-long-horizon program-reconstruction tasks.
Related event: Muse Wonder Model Enters ProgramBench Evaluation(2 posts)→
More from Models
- Meta AI chief boosts claim that Muse Wonder Full Max may be the best model today — alexandr_wang · 2026-10-13
- Zuckerberg: subscribe today to power your Muse with Muse Wonder — iamaliveix · 2026-10-13
- Nace AI's Drex 2.0: a 35B decision-only model with 3B active params beats Jev on Decision Index — techNmak · 2026-10-13
- Musk says xAI's Digital Optimus beats half of Diablo's campaign by watching the screen — RebeccaBellan · 2026-10-13
- Meta's Alexandr Wang confirms Muse Code quota bug, promises usage resets — alexandr_wang · 2026-10-13
- Muse Spark subscribers hit quota limits within minutes of launch — MatthewBerman · 2026-10-13