Vercel: a simple file system beat their multi-agent system at 2x eval score
MaryamMiradi · x · 2026-11-11
Summarizing Vercel's AI Engineer talk "How We Solved Agent Building": after multiple architecture attempts, a simple file system approach beat their elaborate multi-agent system — file system + Bash roughly doubled eval scores. The author maps this into 6 evolution stages and 19 repos:
- Stage 1: One Big Prompt — Vercel AI SDK, LiteLLM, Instructor; first test if the LLM can generate valid SQL
- Stage 2: Multi-Agent System — LangGraph, CrewAI, Google ADK, OpenAI Agents SDK; specialized but context gets lost in handoffs
- Stage 3: One Stateful Agent — PydanticAI, Deep Agents, Mastra; plan, execute, and recover from errors
- Stage 4: File System Agent — Claude Code, just-bash, Vercel Sandbox; the big win
- Stage 5: Reusable Agent Skills — Anthropic Skills, Skills CLI, Vercel Agent Skills
- Stage 6: Production Frameworks — Eve, Workflow DevKit, OpenTelemetry
Takeaway: more agents ≠ better agents — better context, familiar tools, and a simple file system often beat complex orchestration. The author plans a benchmark of LangGraph vs Google ADK vs PydanticAI vs Deep Agents vs Eve on insurance claims, wind turbine maintenance, and legal contracts.
More from coding & agent
- Nous Research previews Hermes Desktop redesign for desktop and web, ETA 1-2 weeks — NousResearch · 2026-11-11
- Salesforce open-sources Pronto sample app for multi-agent orchestration and Headless UI — msrivastav13 · 2026-11-11
- Liquid AI shares tutorial: build a Discord spam-detection bot with its d1-ultra decision model — helloiamleonie · 2026-11-11
- Obsidian creator: make models replaceable cogs, don't chase the latest LLM — dSebastien · 2026-11-11
- Matt Pocock's "Home Repo" trick: one repo to run AI agents across many codebases — mattpocockuk · 2026-11-11
- Dev builds full dotnetconf Event Wall in a week with Copilot and Azure — DanWahlin · 2026-11-11