Autonomous agent fleet proposes 397 experiments to top ClickBench, 1.2× ahead of next engine
jiayq · x · 2026-10-15
jiayq's team reports that their autonomous agent fleet took #1 on ClickBench — 1.2× faster than the next best engine and 3.1× faster than the codebase it started from, which they believe is the first time an agent fleet topped a public systems benchmark with independently verified correctness.
- The fleet proposed 397 experiments, each with a design, independent review, correctness gates, and timing against the best build
- 98 experiments produced improvements; 289 were retired with written reasons, and all outcomes went into a shared record read first by the next proposal
- The team stripped out benchmark-only optimizations ("AI feature creeps"); their entry also ranks first on a concurrent QPS measure added later, which the fleet never targeted
- Takeaway: as agents improve, surrounding infrastructure becomes the main bottleneck, and letting the fleet improve its own methodology paid off; the engine will be open sourced
More from coding & agent
- Debugging 3 bugs reveals what engineers are really worth in the AI era — rseroter · 2026-10-15
- Epic's vibe-coded launcher gets Community Noted over security risks — mark_k · 2026-10-15
- Gemini 4 Argon flubs multi-agent demo: loses conversation track after 4-5 prompts — gaganghotra_ · 2026-10-15
- Dev launches remote MCP server where agents share tested workflow recipes — L3gitCake · 2026-10-15
- OpenAI's Amp adds Puck voice assistant with GPT-Live-1, tests Ultrafast with GPT-6 Astra — Codeblix_Ltd · 2026-10-15
- 56% of US travelers already plan trips with AI — agentic tourism becomes a new distribution channel — robertonappe · 2026-10-15