Testing LLM routing: decision-model delegation pays off only in big harnesses
yb2698 · x · 2026-10-23
The author tests the community idea of using a decision model to route queries to a cheaper small model each turn: Qwen3.5-27B as the large model with vLLM d3 deciding when to delegate to Qwen3.5-9B. Harnesses tested include ipython, Pi, Claude Code and Qwen-Code; the small model runs pure ipython. 10 hard TB-Lite tasks, each repeated 3 times.
More from coding & agent
- Vercel adds no-account deploys: drag and drop a file to get a live URL — steren · 2026-10-23
- Dev runs 5 Claude Code sessions that coordinate via MCP Agent Mail to root his network cameras — doodlestein · 2026-10-23
- Vercel ships 'vercel deploy --temporary' for ephemeral agent-built deployments — cramforce · 2026-10-23
- Running multiple agents out of one Notion workspace as a shared task board — nbaschez · 2026-10-23
- Sierra Summit London feels underwhelming vs SF; production agent metrics still rare — DimlyDimwitted · 2026-10-23
- C-Sweet: open-source business simulator runs a company on local 27B models — Demonicated · 2026-10-23