vLLM Semantic Router makes Mixture-of-Models programmable; evals show instructions swing agent MCP tool use from 0/40 to 20/40
colinmcnamara · x · 2026-10-06
Colin McNamara shares projects from his AIMUG talk: LangChain, vLLM, and the vLLM Semantic Router — a programmable Mixture-of-Models router that sends each request to the best-fit model (Kimi, Claude, Gemini, DeepSeek, Qwen, etc.), backed by quotes from Satya Nadella, Jensen Huang and others on the multi-model routing trend.
Also at the meetup: Jeff Linwood ran 120 Harbor evals on how coding-agent instructions affect MCP tool use. With no instructions, 0 of 40 runs finished the task workflow; a short instruction got 20/40; fully instructed runs passed all 120 coding tests. Instruction quality turns out to be decisive for whether agents complete their tool-call chains.
More from coding & agent
- What a Jumping Spider With 600,000 Neurons Teaches Us About AI Agent Orchestration — uxmag · 2026-10-06
- AI Estimates 5-8 Weeks of Engineering; Developer Has Sub-Agents Finish It in Two Days — Dan_Jeffries1 · 2026-10-06
- AI formal verification hits reality: the halting problem blocks existing codebases — danbri · 2026-10-06
- Developer Builds SEO/AEO Feedback Loop with Grok Bot and Claude Code — doooyle · 2026-10-06
- Next experiment: running most daily todos entirely through voice-driven Grok Bot — FinanceYF5 · 2026-10-06
- Voice-controlled Grok Bot handles Notion, Slack, email and even payments — FinanceYF5 · 2026-10-06