Dynamic model routing loses to guardrails: 120-run test shows strong models are cheaper
shane-testronaut · reddit · 2026-10-11
The author built a reasoning layer with Jev for an agentic browser-testing system to test whether dynamic model routing saves money — and the results flipped the hypothesis:
- The strongest result came from a fixed execution model with guardrails: guarded GPT-5.5 completed 15/15 missions while using 13.7% fewer tokens than the unguarded Terra control.
- Cross-provider dynamic selection across 120 isolated missions: the routing decision cost only 621–724 tokens per turn, but dynamically routed setups showed no overall token advantage.
Key insight: instead of asking "what's the cheapest model for this step," ask "what's the cheapest successful trajectory through the whole task" — a stronger model that avoids retries, bad decisions, and needless exploration ends up cheaper. Methodology, per-condition data, and caveats are published.
More from coding & agent
- Skip the orchestration layer: let your AI agents email each other — NickPassig · 2026-10-11
- Husband uses Claude Code to write a CLAUDE.md guardrail file for his wife's projects — ColleenMBrady · 2026-10-11
- Dev registers haoyong.si and turns server management into a one-sentence skill — vista8 · 2026-10-11
- Action Hub: A Meta-MCP Server That Cuts 90k Tokens of Tool Schemas to 600 — Leading-Pudding-1841 · 2026-10-11
- Open-source repo ships 17 public skills for Codex and Claude to run LinkedIn content ops — tom_doerr · 2026-10-11
- Where AI agent sessions waste tokens: six levers and 30 fixes — blaizedsouza · 2026-10-11