vLLM open-sources System Duo: a fast/slow semantic router that only calls deep reasoning when needed
vllm_project · x · 2026-11-06
The vLLM project announced System Duo, built by Xunzhuo Liu and the vLLM Semantic Router team — a dual-system architecture inspired by Thinking, Fast and Slow.
- System One: fast decision models handle blink-of-an-eye calls — triage, guardrails, verification — with probability output for every option;
- System Two: the router sends requests to deep reasoning models only when it matters, picking both the model and the effort level;
- Fully open source, running on vLLM, with a hosted API at app.vllm-sr.ai (free key on first run, research use only). Demo cases include support-ticket routing (billing vs engineering) and a DeepSeek Harness coding-agent sandbox.
A practical 'one router, two minds' pattern for making agents cheaper and faster.
More from coding & agent
- Chalk talk: Jediah Katz gives an inside look at their agent harness design — jxnlco · 2026-11-06
- Tinker team benchmarks model finetunability: Inkling 2 Small learns faster for less — ziqiao_ma · 2026-11-06
- 28,849-trial Harness Leaderboard: bash-only mini-swe-agent beats openclaw 51.4% vs 27.3% — bansalg_ · 2026-11-06
- Qwen 27B + Python REPL harness hits 78.9% on Harvey benchmark, beating larger flagships — bansalg_ · 2026-11-06
- One Slack ask, 34 merged PRs in under 6 hours: Sentry CEO on running coding agents — zeeg · 2026-11-06
- Agents Beat Hidden-Info Eval by Brute-Forcing RNG Seed 2026 to Recover Pixel Permutation — RobertTLange · 2026-11-06