Video: Building a Local LLM Routing Layer with NVIDIA SwitchYard
Prompt Engineering · youtube · 2026-08-11
This video explains why a routing layer is essential for agentic systems and demos how to run a custom router locally using NVIDIA SwitchYard to balance cost, speed, specialization, and privacy.
- Routing Options: Compares OpenRouter, Devin Fusion, and RouteLLM. Highlights NVIDIA Switchyard (built on RouteLLM), supporting random, LLM classifier, stage routing, and escalation strategies.
- Models & Performance: Introduces NVIDIA Nemotron 3.5 Lightning (30B latent MoE), showcasing its high throughput advantage with a speed test against Kimi K3.
- Workflow Demo: Demos a local-first alert analysis workflow where Nemotron generates private digests, escalating to Kimi K3 only when needed via Switchyard.
- Best Practices: Advises that routing should be executed per task/session rather than per turn.
Related event: NVIDIA Unveils Nemotron 3.5 Lightning and NeMo Switchyard Router(55 posts)→
More from coding & agent
- Meta open-sources Muse Spark Open agent model with day-0 TPU support in SGLang — BanghuaZ · 2026-10-16
- Saying "load-bearing" in your prompt makes Opus 5.5 stop holding back — zacharynado · 2026-10-16
- You.com partners with Paperclip to give every agent live web access — PolarBearby · 2026-10-16
- I priced every Grok Bot call: $20 to $200 plan in 5 days, long jobs eat 52% — jonathanmalkin · 2026-10-16
- GitHub is criminally underrated as a free data store, devs argue — blelbach · 2026-10-16
- Claude Desktop doesn't support MCP sampling — here's a workaround for two-way MCP apps — wuj · 2026-10-16