Agent Evaluation Science Symposium CFP: coding-agent teams at 70% lower cost teased
nouhadziri · x · 2026-10-15
The Agent Evaluation Science Symposium (NYC, Nov 20) is accepting 2-page extended abstracts and short papers on agent evaluation; published work is welcome. Abstract deadline Oct 20, paper deadline Oct 25, with free registration and travel support.
Scope covers four tracks: Diagnose (audits, replications, negative results, contamination), Measure (constructs, metrics, human evaluation, automated graders), Operationalize (benchmarks, environments, traces, scalable grading), and Apply (coding, research, healthcare, enterprise tool-using agents).
The poster also previews their own talk: the science of scaling teams of coding agents—when teams outperform single agents, where collaboration pays off most, and how smarter compute allocation and heterogeneous teams push the performance-cost frontier, claiming near-frontier team performance at 70% lower cost, with the paper and blog dropping soon.
Related event: Agent Evaluation Science Symposium Calls for Papers, Deadline Oct 20(2 posts)→
More from coding & agent
- Ditching DaVinci and Premiere: editing video entirely in Codex or Claude Code — MatthewBerman · 2026-10-23
- Grok Bot upgrade lets xAI's agent send messages, manage calendars and control smart home on your phone — XFreeze · 2026-10-23
- Open-source AI video previz tool CozyClay gets a deep-dive demo — cocktailpeanut · 2026-10-23
- Voice AI agent operator's week of disasters: misdialing Russia, bots reporting success while broken — ArseniyZ83 · 2026-10-23
- Supabase adds one-command MCP servers so ChatGPT and Claude can drive your app — dshukertjr · 2026-10-23
- Grok Bot ships with its own cloud computer: browser, terminal and filesystem to finish tasks — nima_owji · 2026-10-23