One 4B router unifies single-, multi-round, and agentic LLM routing; 61.4% avg, +11% over best

RouteWeaver: Weaving Mode Selection and Execution into Unified LLM Routing

Xiaohan Wang, Haozhen Zhang, Qingyuan Liu, Tao Feng, Wenya Wang

cs.CL

2026-10-14

RouteWeaver trains a single 4B router to pick single-round, multi-round, or agentic execution per query, averaging 61.4% across five domains, +11.03% over the strongest baseline.

What problem this solves

LLM routing decides which model handles each query. Existing routers fall into three camps by how they organize calls: single-round routers (GraphRouter) pick one model for one call; multi-round routers (Router-R1) make successive picks conditioned on intermediate outputs; agentic routers (MasRouter, GraphPlanner) arrange models into workflows with roles and dependencies. In all three, the paradigm is fixed in advance. The router can adapt which models it uses, never how the calls should be organized for the query at hand.

The premise checks out in the paper's own numbers: across five evaluation domains the strongest fixed-paradigm baseline differs, with multi-round routers winning Math, Code, and Knowledge while single-round routers win Reason and Recall. No paradigm dominates, so the paradigm itself should be part of the routing decision.

Method

RouteWeaver folds the three paradigms into one LLM policy. Three pieces:

• A shared action grammar. The router, a fully fine-tuned Qwen3-4B-Instruct-2507, first emits a mode tag (single/multi/agentic), then generates calls under that mode: each call names a Worker by anonymous ID plus a sub-query, and agentic calls add a role (planner/executor/verifier/summarizer) plus references to earlier outputs. Worker responses return as observations appended to the router's context until it emits a final answer. The pool holds six frozen models, from Qwen3-8B to Gemini-2.5-FlashLite, exposed only as anonymous IDs with capability descriptions; the router never sees names or prices. A trajectory that breaks the grammar or mode constraints terminates on the spot with a -1 reward.
• COMET-GRPO. Plain GRPO assigns one group-relative advantage to every token of a trajectory, so it cannot separate a wrong mode choice from poor execution inside a right one. COMET-GRPO splits the signal in two: Amode measures how the selected mode's group mean compares with the overall group mean, asking whether this mode suits this query on average; Ainner measures how the trajectory compares with same-mode peers, asking whether the execution was good. The first lands only on mode-selection tokens, the second on the remaining router tokens, each loss normalized by its own effective token count so the short mode decision is not drowned out. Task and efficiency rewards (Worker call cost at fixed reference prices) are normalized separately and mixed with weight α.
• Progressive exploration. Free mode choice from step one starves unfavored modes of training signal. Steps 1-25 force an assigned mode on every query and train plain GRPO on task reward only; steps 25-75 linearly decay the forced ratio to zero while a support penalty keeps probability mass on all three modes; after step 75 the router is fully autonomous under COMET-GRPO. Learn how to execute each mode first, then learn when to use it.

Results

Nine benchmarks, five domains, 1,060 test questions disjoint from training, main results with α = 0:

That is 61.4% on average, an 11.03% relative gain over Router-R1, first place in all five domains; the margins over the best single-round and agentic baselines reach 26.08% and 31.20%. The learned mode mix splits by task: 99% of Reason queries go agentic, 96% of Recall goes single-round, Code is 84% multi-round, Knowledge 72% single-round, and Math uses all three (19/50/31).

Three further results:

• Cost dial. At α = 0.1, Worker cost drops 23% for a 1.0-point accuracy loss; at α = 0.5 cost drops 86% but accuracy falls 11.2 points, where the trade turns bad.
• Ablations. Swapping in plain GRPO drops the average to 58.8% and skews mode usage to .13/.82/.05; removing progressive exploration drops it to 55.6% with the policy collapsed to 100% single-round. That collapsed variant still edges the best baseline's 55.3%.
• Held-out generalization. On AIME, GPQA-Diamond, and BBH, none used in training, RouteWeaver averages 53.7% against Router-R1's 45.4%, an 18.28% relative gain, leading by 11.4, 7.1, and 6.4 points.

Why it matters

Teams building model gateways can take two things away. The execution paradigm can be learned per query rather than fixed at design time, and a 4B policy suffices to switch between one cheap call and an eight-call workflow. α then gives cost operating points on the same trained policy, with no retraining for a cheaper tier. The progressive-exploration lesson generalizes to any multi-mode agent training: feed each mode enough execution experience before letting the policy choose, or it collapses early. Code is on GitHub.

The comparison set is a batch of fixed-paradigm routers, and the absolute numbers depend on this six-model pool; conclusions may not transfer unchanged to a different pool.

Limitations

Boundaries the authors state: agentic mode has no tool use, four fixed roles, at most eight Worker calls; richer execution structures and tool-augmented environments are left to future work, and training uses only 3,200 queries.

Grounds for skepticism: each test question gets a single system rollout with Workers sampling at temperature 0.2, and the authors concede evaluation is not fully deterministic, so one-point margins deserve slack. Cost accounting covers Worker calls at fixed reference prices and excludes the router's own compute and orchestration overhead. Whether all ten baselines share the same Worker pool and training data is not stated in the main text. The collapse ablation also shows something the headline misses: a fully collapsed single-round policy still scores 55.6%, nearly tying Router-R1, so the unified grammar and training recipe alone carry real weight, and the increment from actual mode diversity concentrates in execution-heavy domains like Math and Code.

Terms

• LLM routing: a dispatch layer that assigns each query to models based on strengths and cost.
• Single-/multi-round/agentic routing: one model, one call; successive calls conditioned on intermediate outputs; models organized into a workflow with roles and dependencies.
• GRPO: Group Relative Policy Optimization; advantages come from the mean and std of a group of rollouts per query, no critic needed.
• Credit assignment: attributing a trajectory's final reward to individual decisions along the way, here mode choice versus in-mode execution.
• Worker: a candidate model the router can call; six frozen models exposed under anonymous IDs.

Source

• arXiv 2610.19705v3
• PDF
• Hugging Face
• GitHub · https://github.com/LaughKing/RouteWeaver

Related papers

• EvoForesight leads FutureX three weeks running, with rating gains up to 195%
• Heartbeat finishes tool calls mid-speech: FDBench coverage +16 pts, latency -36.4%
• IQuest-Q1: an open 320B MoE for CLI agents that posts 84.5 on CyberGym, ahead of GPT-5.6 Sol
• Chronoberg: a period-synonym swap flips 50-75% of hate calls in 3 of 4 LLMs
• Chunk-level DFM: 32× fewer queries, 12.71% above the best Mistral QA baseline
• EinsteinArena: 11 new math records by May 2026, kissing number from 593 to 604

All paper explainers