Community autoresearch coding benchmark: open models underwhelm

Josh Purtell's community autoresearch coding benchmark shows open-source models underwhelming and Muse Open far below promises; closed-source coverage is limited to OpenAI models because running Claude would violate Anthropic's terms.

2026-11-07 ~ 2026-11-07 · 2 related posts

• Autoresearch coding leaderboard: best open model leads but underwhelms, muse open far below promised 1.2 — JoshPurtell · 2026-11-07
• Using Claude Subscriptions for Evals Likely Violates Anthropic's ToS, Bench Author Says — JoshPurtell · 2026-11-07