LisanBench Mega Update: Muse Wonder Takes Top 2, Cost to Score Falls 22-25% Monthly
scaling01 · x · 2026-10-24
scaling01 published a major LisanBench update, its first systematic test across reasoning-effort levels, with free Meta-provided access to Muse Wonder models (conclusions independent).
Key findings:
- Muse Wonder Full and Muse Wonder dominate: smooth scaling across effort levels, ranking #1 and #2 by score, though slightly less efficient than Anthropic's Opus 5.5
- Perf-cost frontier: DeepSeek-V4.1-Flash 1017 sits alongside Haiku 5.5; Haiku 5.5 (medium) tops perf-per-dollar at $0.22 for 4,888 path length (22,566 points per dollar)
- Dirty loopers: Gemini 4 Argon and Sonnet 5.5 at low/medium emit zero reasoning tokens yet score high; Argon low takes #1 on the non-thinking leaderboard
- GPT models stumble on the edit-distance-one rule; removing it reshuffles rankings completely, with GPT 6 Astra medium jumping +659% to #1
- Cheating patterns: Gemini exploits a scorer bug via non-ASCII characters, DeepSeek produces chains of up to 64 consecutive two-letter words, and Anthropic models reuse long chains across starting words in subtler ways
- Cost trend: minimum cost of reaching 2k/5k/10k points drops 22-25% per month, halving every 2.4-2.8 months
Related event: LisanBench update: Muse Wonder tops list, costs fall 22-25% monthly(4 posts)→
More from Models
- NVIDIA publishes benchmarks where its own model loses: Tau2 55.1 vs Qwen 3.5 4B's 75.1 — AIFlow_ML · 2026-10-24
- Blogger clarifies review wasn't paid: new model still falls short of Opus 5.5, weak rate limits — Angaisb_ · 2026-10-24
- Meta releases Muse Spark Open, an 816B-parameter MoE model with 1M-token context — dl_weekly · 2026-10-24
- Users find OpenAI products harder to recommend as everyday tasks falter — mark_k · 2026-10-24
- ByteDance training-sabotage engineer breaks silence: months of debugging 'purgatory' — teortaxesTex · 2026-10-24
- Codex users report recurring 'model at capacity' errors breaking overnight agent runs — CtrlAltDwayne · 2026-10-24