Jia Yangqing: agent fleet tops ClickBench and concurrent QPS, engine to be open-sourced
jiayq · x · 2026-10-15
Jia Yangqing shares lessons from letting an agent fleet compete on ClickBench: they stripped out any benchmark-specific optimizations ("AI feature creeps"), with reviewer agents removing special-case hacks. When ClickBench later added a concurrent QPS measure the fleet never targeted, their entry ranked first there too. Key takeaway: as agents get more capable, the surrounding infrastructure becomes the main limit on iteration speed, and letting the fleet develop its own methodology paid off. The engine will be open-sourced.
Related event: jiayq's Autonomous Agent Fleet Tops ClickBench, Engine to Be Open-Sourced(6 posts)→
More from coding & agent
- How Codex Happened: Inside OpenAI's coding comeback from a failed launch — every · 2026-10-16
- Claude counts tokens, not messages: 10 tricks to make your usage limit last all day — Aiden_Tech_Ai · 2026-10-16
- How to convert 20-year-tenure engineers to AI-native development before deciding who to retrain — alex_verem · 2026-10-16
- LangChain revamps human-in-the-loop middleware with typed schemas for tool interrupts — Hacubu · 2026-10-16
- Robot fully controlled by frontier LLMs assembles gadgets autonomously for 1.5 hours error-free — ihorbeaver · 2026-10-16
- Why the future of agents is basically Node and JavaScript, in one argument — verrsane · 2026-10-16