Google DeepMind measures whether model routers are stable, not just accurate
burkov · x · 2026-07-21
A new Google DeepMind paper studies how to route requests among multiple language models or agents.
The authors define two measurements: one captures how differently models behave across tasks, and the other tests whether meaning-preserving edits such as typos, paraphrases, or irrelevant additions keep routing decisions stable. Their experiments suggest that a carefully chosen set of fewer than ten models can preserve most useful behavioral diversity, while nearest-neighbor routers may be accurate yet brittle under small wording changes. Routers based on model descriptions can be less accurate in some settings but much more stable.
Related event: Google DeepMind Proposes Metrics for Model Routing Stability(2 posts)→
More from Research
- EMNLP Oral: training-free Nexus Sampling cuts KV-cache memory 10x; OpenResearcher tops BrowseComp-Plus at 54.8% — yuz9yuz · 2026-10-27
- ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
- Forking gap follows a power law in context frequency; masking rare n-grams shrinks it — ZimingLiu11 · 2026-10-27
- How forking happens: n-gram tables split logits into non-interacting subspaces — ZimingLiu11 · 2026-10-27
- The best LoRA explainer video yet: rank constraints meet intrinsic dimension — sophiamyang · 2026-10-27
- No π0 weights changed: Motoniq layer lifts real-robot precision fitting from 0/10 to 9/10 — hbouammar · 2026-10-27