Replit's ViBench 2.0: best models pass only 49–65% of builds, GPT-6.1 Sol leads
amasad · x · 2026-10-29
Replit released ViBench 2.0, a prompt-to-app benchmark. GPT-6.1 Sol leads with 65% pass@1 at the lowest cost ($10/build); Opus 5.5 hits 48%; GPT-6 Astra reaches 60% but costs $85/build. Sonnet 5.5 manages 19%, while DeepSeek V4.1 Flash, Gemini 3.8 Flash, GLM 5.3, Kimi K3 and Grok 4.7 fall below 2–5%. Mean of 4 builds per 17 products, 95% intervals.
Related event: Replit's ViBench 2.0: Best Models Pass Only 49-65% of Build Tasks(2 posts)→
More from coding & agent
- LangChain engineer: most agent rules made it worse — deleted them after Opus 5.5 — LangChain · 2026-10-29
- His most-starred repo is now fully vibecoded and vibe-PR'd — LFAN passed in 1.5 GPU-hours — ChongZzZhang · 2026-10-29
- Vercel Launches Agentic CI With Turborepo Task Skipping, 2-6x Faster Builds — shuding · 2026-10-29
- Celesto AI lets you run GitHub Actions on its cloud computers with per-job pricing — aniketmaurya · 2026-10-29
- GitHub sees 7.4B monthly commits from AI agents, adds 4M CPU cores to cope — ai · 2026-10-29
- 5 practical tips to cut CI time when coding with AI agents — aniketmaurya · 2026-10-29