Largest systematic agentic eval campaign launches: test agents as tool users, not tools
animesh_garg · x · 2026-11-14
Nicolas Keller's team announced the largest systematic eval campaign for agents, arguing most benchmarks wrongly treat agents as tools rather than intelligent tool users — likening the shift to Homo habilis's leap via tool use. Retransmitted by Animesh Garg. Full details in the original 12-post thread.
More from coding & agent
- AmphoraDB launches serverless databases for agent swarms, paid per statement in stablecoins — banteg · 2026-11-14
- Didit now offers identity checks via MCP — bosmeny · 2026-11-14
- Ex-quant's local AI agent negotiated 1,000+ influencer outreach in 9 days — Shruti_0810 · 2026-11-14
- With 90% AI-generated code, Limestone gates merges on its V.U.E. three checks — alex_verem · 2026-11-14
- Ford dealership veteran pays for an AI system doing the work of 12 salespeople — aryanXmahajan · 2026-11-14
- One prompt gets AI to integrate WebMIDI so you can play your MIDI keyboard in the browser — alexanderchen · 2026-11-14