GLM 5.3 Flash jumps 29% to 82% on AppWorld after Beaker tunes the agent harness

VictorSanh · x · 2026-11-17

Beaker's team argues that agent harnesses and prompts are usually built for one model, so swapping models unfairly penalizes them. On the AppWorld coding-agent benchmark, GLM 5.3 Flash completed just 29% of scenarios untuned — but after Beaker autonomously tuned the agent for it, it scored 82% at nearly identical cost ($0.029→$0.033 per scenario) and almost 2x speed (138s→76s). Across eight models, Beaker's per-model optimization turned initially weak models competitive, showing the two strongest models needed completely different adjustments and reframing model choice as a tradeoff between quality, cost, and latency rather than leaderboard position.

Related event: Beaker: Tune Your Agent Harness Before Comparing Models(2 posts)→

Original post →

More from coding & agent

coding & agent channel →