Microsoft paper: history-aware model routing lifts multi-turn agent task pass rate by 3.3 points
dair_ai · x · 2026-10-16
A new Microsoft paper, Whose Turn Is It? Context-Guided Model Routing for Stateful Agents, studies zero-shot, cost-aware model routing at the user-turn boundary for multi-turn agents.
- Key trick: give the router the session history, not just the current request. At a matched offloading rate, the history-aware router passes 3.3 points more tasks on multi-turn spreadsheet agents (Sheet Copilot / SpreadsheetBench 2), with gains for every model pair.
- Cache economics: every model switch invalidates the prompt cache. Switching from the expensive model to the cheap one recovers its cache cost within two LLM calls; switching up never broke even in their tests.
- Another finding: the cheap model's per-task cost advantage shrinks from 6.8x to 3.2x on longer sessions — cost depends on execution dynamics, not just token prices.
Takeaway: agent routing is a sequential decision problem; effective policies must account for session history and the asymmetric cost of switching models.
More from coding & agent
- VibeZero: one-shot an entire vibe coding platform on Cloudflare Void — cnakazawa · 2026-10-16
- Ant and Tsinghua open-source Realtime-Venus, an OpenAI Realtime API alternative — aigclink · 2026-10-16
- 'I remember when I was the harness for o1 pro' — dev on becoming the API for his model — timigod · 2026-10-16
- JevAgentHarness: a 4B decision model lets cheap agents offload tool choices from the LLM — youjiaxuan · 2026-10-16
- Coding agents keep opening 3 stacked PRs per chat, and merging them is a nightmare — holdenmatt · 2026-10-16
- Google-led paper uses sequential auctions to organize decentralized LLM agent workflows — _onionesque · 2026-10-16