Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems
Pengfei Zhou, Yixing Ma, Hexin Wang, Weidong Tang, Zhenglin Wan, Wangbo Zhao, Yang You
cs.CV
2026-11-04
Across 25 benchmarks and 374,401 runs, per-input oracle routing beats the best fixed model by 5.4 points. Learned routers add 0.3; cheaper routes average 8.82× the reference.
A general-purpose system has to answer, write code, draw, render video, speak, and move a robot. A candidate is one item on that menu: a language model, a generator, a robot policy, or a whole plan that tries a cheap model and escalates when verification fails.
The best item changes with the input. Pinning each benchmark to its strongest candidate still misses on individual examples. Routing work has mostly stayed inside language models, escalating on failure or training a dispatcher on preferences. Video and robot pools have no candidate that wins everywhere. Speech already sits near the ceiling. The missing piece was a shared-input score matrix across those output types.
Route Anything plays, in the state it can see, the candidate with the highest expected score minus price times expected resource use. The state may be only the benchmark name, the full input, a capability type, or three camera frames plus the robot state. With no prices it plays the highest expected score. Name a reference policy and a cap on the quality drop. The route that maximizes efficiency inside that floor is the same rule at one price. Efficiency means quality per dollar, or per second of policy inference on a robot.
How far the router can go is an information ladder. The coarsest rung freezes the strongest training candidate on each benchmark. A finer rung picks by capability type. The per-input oracle then picks whoever actually scored highest on that input. It is a ceiling, not a call you can make at deployment. Above it sits outcome noise: given the input, the winner can still be a coin flip. On AA-LCR every question is answered three times, and the oracle's 2.4-point lead over the fixed candidate disappears once each candidate is scored by its three-run mean.
The training target follows that ladder. Regress each candidate's own normalized score, then play the highest prediction. Classifying the winner throws ties away. On the robot training set a latency tie-break sets 94% of winner labels, and 24,556 of 31,408 successful outcomes are marked as the wrong class. The classifier learns to pick fast policies that often fail.
The measurements are one omni-modal trajectory matrix: 25 benchmarks, 452 candidate configurations, 374,401 scored runs. Seven groups cover text QA, image-text QA, image, video, and speech generation, code and workplace agents, and bimanual manipulation, with three to seven candidates each. Language-model candidates are Claude Opus 5.5, GPT-6.1 Sol, and DeepSeek V4.1 Flash, with GPT-6 Astra added on long-context, knowledge, and agent tasks. Inputs are split 70/10/20 by source. RoboTwin's 2,000 test scenes come from 10 tasks the router never saw. Routers fine-tune Qwen3.5-4B. A capability-tree router assigns the input to one of up to 157 leaves and looks up that leaf's best training candidate, falling back to the fixed candidate when it cannot place the input. The robot router also reads three 240×320 frames and predicts success for each of seven vision-language-action policies. A VLA maps camera frames and an instruction to actions.
On the 19,814 test inputs where every candidate has a score, across 24 benchmarks, the fixed candidate chosen on training scores 82.60. Re-picking the best candidate per benchmark on the test set itself reaches only 82.62. The per-input oracle scores 87.98, 5.4 points higher.
| Group | Fixed | Per-input oracle | Gap |
| Text QA | 83.27 | 87.56 | 4.28 |
| Image-text QA | 85.32 | 88.02 | 2.70 |
| Image generation | 82.93 | 87.92 | 4.99 |
| Video generation | 62.43 | 81.36 | 18.93 |
| Speech generation | 98.62 | 99.30 | 0.68 |
| Code and workplace | 69.17 | 77.29 | 8.11 |
| Robots | 70.45 | 88.85 | 18.40 |
On the 18 benchmarks that carry capability labels, the best candidate per type, even chosen in hindsight, recovers 8% of the gap. On RoboTwin, the best policy per task and scene condition recovers 23% of the 18.4-point gap.
One capability-tree router scores 79.98 on 3,444 test inputs from 16 benchmarks, against 79.68 for the fixed candidate and 84.80 for the oracle. The extra 0.30 point is about 6% of that ceiling. Video moves from 62.17 to 64.28. Code and workplace agents add 0.77, image generation adds 0.25, text QA loses 0.17, and speech is flat. The concentrated moves are Terminal-Bench at +8.3 on 36 inputs and VBench-2.0 at +2.1; the largest loss is AA-LCR at −2.4. The leaf classifier is 86% to 89% accurate, yet 83% to 86% of inputs still go to the fixed candidate. Across 124 configurations on GenEval2 and Qwen-Image-Bench, none beats the fixed generator with an interval above zero, and 44 are significantly worse. An untrained Qwen3.5-4B that names the candidate itself lands 1.15 points below the fixed one.
On robots the outcome router reaches 71.55% success, against 70.45% for the best single policy, ME-Dex. The difference is +1.10, interval [-0.40, +2.55]. It sends 929 of 1,000 clean scenes to PatchWAM and 842 of 1,000 randomized scenes to ME-Dex. Policy time drops from 14.51 to 11.05 seconds per scene, 24% less. A winner-label classifier under the same setup reaches 42.8%.
Mean efficiency over four testbeds is 8.82× the reference, the average of four ratios. Code against always calling Opus 4.6 is 25.5× at a 0.44-point loss, interval [-1.75, +0.97], and that zero-loss fit never calls Opus. Text against always calling GPT-4-1106-preview is 7.1× at a 0.40-point loss. Avoiding the reference entirely costs 9.7 points on that bench, so the zero-loss text route still escalates. GPT Image 2.5 costs $0.21 an image, 3.5× to 5.4× the other generators. A half-point tolerance cuts image test cost by 26%, which is 1.35×, at a 0.47-point loss. Robots are 1.33×. A fitted floor is not a test guarantee: refit on half the probing tasks and it holds on test in 12.5% of 200 splits. Certify it on the other half and it holds in all 200, at a median 19.4×. Missing scores flip the conclusion. Opus 5.5 lacks 11 of 1,122 HLE training scores, the pooled fixed baseline falls from 68.30 to 65.52, and the outcome router's 0.40-point deficit becomes a 2.38-point gain.
The headroom is real: about 19 points on video and about 19 on robots. A type label recovers roughly a tenth of it, and a trained router adds 0.3 on the 16-benchmark panel. When quality is close and prices are far apart, the dependable product is a cheaper route, mid-twenties on code and about 7× on text QA. A 4B router takes 0.13 seconds a decision on one H20. An image takes 27 to 158 seconds, so the router overhead is small next to generation.
Three rules are ready to use. Compare candidates on the same inputs. Train on each candidate's score, not on winner labels. Fix the reference and the tolerance, then certify the floor on held-out inputs. Extra spend inside a task is not automatically worth it. On Qwen3-Max a free compiler gate reaches 4.09× dollar efficiency at a 2.4-point loss. An LLM judge is 2.63× if it is free and 0.30× at list price.
The 0.3-point quality gain does not establish a general orchestrator. The cost result, on code and on text, does.
The learned quality gains are soft. The +0.3 uses one training seed. On 11 benchmarks and four seeds the outcome router is 0.40 behind the matched fixed candidate, interval [-1.19, +0.33]. The robot interval crosses zero. None of the 124 image configurations wins significantly. Seven understanding benchmarks, about three quarters of the inputs and about half the pooled oracle gap, sit outside the learned-router evaluation. Their gaps are 1.4 to 6.0 points. They are MMLU-Pro, MMLU-Redux, SuperGPQA, GPQA, and three image-text QA sets. Every benchmark with a gap above 8 points was kept in the learning study, so the easier understanding tasks were the ones left out.
Part of the 5.4 is noise that a repeated grade removes. Most of the matrix records no cost, so the quality ladder and the efficiency ratios come from different benches. The 8.82× figure is the arithmetic mean of 25.5, 7.1, 1.35, and 1.33, and the code reference is Opus 4.6, not the Opus 5.5 in the omni-modal pool. SciCode is excluded because later steps contain the model's own earlier code. The robot split is unseen tasks inside one manipulation benchmark, not a second suite. Certification buys a real floor and gives back some of the fitted code ratio, from 25.5× to a median 19.4×. That certificate is not applied to the video or image quality gains.