Senior Engineer Benchmark: Muse scores 42% rewriting slop codebases vs 71% for Sol 6.1
danshipper · x · 2026-10-13
danshipper published the public scoreboard for his Senior Engineer Benchmark, which asks models to rewrite a vibe-coded slop codebase from scratch:
- gpt-6.1-sol (Codex CLI, high effort): 71%
- muse-wonder-1.0: 42%
- gpt-6-sol: 35%
It's a single-person benchmark built on real work, scored by programmatic code checks plus an LLM judge (Claude), answering "which model is best at my job" rather than making a general quality claim. Sol 6.1 is itself not the top-scoring model, and scores for other models are pending.
Related event: Muse Wonder Lags Far Behind Sol 6.1 on Senior Engineer Benchmark(3 posts)→
More from Models
- Meta's Alexandr Wang says natural communication was a big focus for Muse Wonder — alexandr_wang · 2026-10-13
- Dev says Opus 5.5 now feels sloppy and needs hand-holding like every other LLM — audrow · 2026-10-13
- Decision models: the new Hub category that outputs numbers, not paragraphs — erhannah · 2026-10-13
- Muse Wonder Video Model Tested: 'Pretty Rough' Output, Every Narration in Australian Accent — danshipper · 2026-10-13
- Top 10 AI models' share of developer tokens plunges from 55% to 14% — GaryMarcus · 2026-10-13
- DeepSeek V4.1 Flash suspected silent update shows high code-edit failure rate — teortaxesTex · 2026-10-13