Auto-tuning an AppWorld agent's prompts and tools: 36.6% faster, 34.6% cheaper
VictorSanh · x · 2026-11-13
The author used @withbeaker to autonomously test changes to an agent's prompts and tool definitions. With the model unchanged, the AppWorld agent ran 36.6% faster and cut model cost by 34.6% while keeping a similar goal-completion score.
Changes included fewer repeated lookups, shorter instructions and less unnecessary context. The optimized agent was checked on tasks not used during optimization; the whole run from baseline evaluation to final results took 2h24m. The author wrote up the changes, results and full run.
More from coding & agent
- Omnigent 0.22.0 Adds Schedule and Event Triggers for Mixed Coding-Agent Software Factories — matei_zaharia · 2026-11-14
- Vertskim: Local MCP Server Gives Coding Agents a 3D Codebase Knowledge Graph — Queasy_Debate3423 · 2026-11-14
- Nursa's accountant and retired parole officer built real tools with AI — every · 2026-11-14
- We don't need more chatbots: an agent harness lead's field notes — threepointone · 2026-11-14
- Nexus launches an AI context engine that turns deployment knowledge into agent-queryable graphs — rohanpaul_ai · 2026-11-14
- Developer now runs 100% of his agents locally on a Mac Mini, up from 25% earlier this year — josh_bickett · 2026-11-14