Evolutionary search auto-generates agent tool-use tasks, lifting AppWorld hard-task share to 52.5%
minsuk_chang · x · 2026-10-24
A team including Natasha Jaques and Min Suk Chang proposes auto-generating challenging tool-use tasks via multi-agent evolutionary search over existing toolsets and task primitives, addressing the cost and saturation of human-designed agent evals. The approach raises the share of hard tasks on AppWorld from 19.8% to 52.5%, improves semantic and functional diversity, and keeps tasks realistic (e.g. 'email my roommate'), and can be used for failure diagnosis.
Related event: TURTLE: Evolutionary Search Auto-Generates Hard Tool-Use Benchmarks(2 posts)→
More from coding & agent
- Building a private AI agent for under $20/month: old laptop plus DeepSeek via OpenRouter — lawandd · 2026-10-24
- Codex slow? It's re-running your full test suite after every tiny fix — Some-Following-392 · 2026-10-24
- Context Monorepos: why devs are co-locating docs, decisions and code to make long-running agents viable — _AustinCalvert_ · 2026-10-24
- Screenshot shows multi-agent AI team with a Chief of Staff that self-corrects — RachelVT42 · 2026-10-24
- This all-in-one ESP32 pocket dev board is programmable by Codex and fits on a keychain — thesephist · 2026-10-24
- Desktop ComfyUI was unusable on phones, so he built a free open-source mobile UI — Deneme123deneme · 2026-10-24