Beaker auto-optimizes agents: 36.6% faster and 34.6% cheaper on AppWorld without changing models
VictorSanh · x · 2026-11-13
Beaker demonstrates agent execution speedup without changing the model — it automatically optimizes the agent's prompts and tools to cut wasted work.
AppWorld benchmark results:
- 36.6% faster execution and 34.6% lower per-scenario model cost, via a fully automated 2h24m optimization run.
- Goal completion essentially unchanged: 87.5% before vs 88.0% after (agent uses GPT-6-Luna to write Python against simulated Spotify/Venmo APIs).
- Example: in one Spotify scenario the agent fetched the same login/recommendation API docs three times (once per sub-task); after optimization it fetched once and reused — model calls dropped from 21 to 8 with all task checks passing.
- The run tested 10 hypotheses, keeping only changes that helped; every hypothesis and code change is publicly inspectable.
Core thesis: most agent work chases accuracy and accepts slow, costly execution for a 0.1% bump; once quality is acceptable, the next step is maximizing speed and cost efficiency without hurting correctness.
More from coding & agent
- GitHub course with 24.5k stars teaches harness engineering for AI coding agents — tom_doerr · 2026-11-14
- Your company 'second brain' is just silos with an AI on top — alex_verem · 2026-11-14
- Modal Endpoints land in Nous Research's Hermes Agent via one-line plugin install — AAAzzam · 2026-11-14
- Cmpnd releases dspy-wsp: $5-10 of GPT-6 edits model weights, lifts every decision model tested — dbreunig · 2026-11-14
- AmphoraDB launches serverless databases for agent swarms, paid per statement in stablecoins — banteg · 2026-11-14
- Ex-quant's local AI agent negotiated 1,000+ influencer outreach in 9 days — Shruti_0810 · 2026-11-14