Beaker autonomously cut an AppWorld agent's latency 36.6% and cost 34.6% with the same model
VictorSanh · x · 2026-11-13
A writeup of autonomously optimizing an AppWorld agent with Beaker: without changing the model, the agent tested prompt and tool changes itself, keeping a similar goal completion score while cutting latency 36.6% and model cost 34.6%.
- Changes kept: fewer repeated lookups, shorter instructions, less unnecessary context
- Validation: checked on tasks never seen during optimization
- Total time from baseline eval to final results: 2h24m
Full run details included, with per-hypothesis latency data (baseline 76.7s down to 49.8s) — a hands-on example of eval-driven agent optimization.
More from coding & agent
- Nursa's accountant and retired parole officer built real tools with AI — every · 2026-11-14
- We don't need more chatbots: an agent harness lead's field notes — threepointone · 2026-11-14
- Nexus launches an AI context engine that turns deployment knowledge into agent-queryable graphs — rohanpaul_ai · 2026-11-14
- Developer now runs 100% of his agents locally on a Mac Mini, up from 25% earlier this year — josh_bickett · 2026-11-14
- Mediabunny ships 400 fixes from an LLM-assisted bug hunt across its 20M-download codebase — Vjeux · 2026-11-14
- Perplexity Teammates screenshot shows agents on Datadog duty, reading PRs overnight — testingcatalog · 2026-11-14