Vercel Releases Next.js AI Agent Eval: Kimi K3 and Claude Tie at the Top
evilrabbit_ · x · 2026-08-01
Vercel has released its AI Agent Evaluations for Next.js, testing major models on code generation and migration tasks based on success rate, average duration, and cost.
- Top Tier: Kimi K3, Claude Fable 5 (high), and Cursor Composer 2.5 all achieved a 92% base success rate. Kimi K3 is highly cost-effective ($0.141), while Cursor Composer 2.5 is the fastest and cheapest ($0.046).
- The AGENTS.md Boost: The benchmark highlights the impact of AGENTS.md, which bundles Next.js documentation for agents. With this context, most models (e.g., GLM 5.2, Grok 4.5) saw their success rates jump to 96%.
- Cost vs. Performance: GPT 5.5 Pro took 771s and cost $18.21 but only achieved an 83% success rate. Open-source and Chinese models demonstrated significant advantages in cost-effectiveness.
More from coding & agent
- DeepMind turns coding agent Antigravity into a science tool, surfacing novel antimicrobial peptide in one session — pushmeet · 2026-10-16
- Cross-review coding workflow: Claude writes the plan, Codex audits it — theteknosaur · 2026-10-16
- Simon Willison: code mode is the future of tool calling, if it can be sandboxed — springrod · 2026-10-16
- Normalizing Claude's schema-free MCP output before it hits downstream n8n nodes — snowingbol · 2026-10-16
- Agent named 'Ilya' reportedly scores 1.1pt higher on benchmarks — daniel_mac8 · 2026-10-16
- A plain notes.md fixed my agent's cross-session amnesia better than chat history — rahul_watertech · 2026-10-16