DigitalOcean Open Sources Preference-Based Model Router
AI Engineer · youtube · 2026-08-22
Archana Kamath and Tyler Gillam present a model routing system based on preferences rather than benchmarks. Demos show session costs of 8 cents vs 25 cents for a single premium model, with the gap widening over time.
Core Concept:
- No single best model exists; the right choice depends on task, system prompt, tools, cost, latency, and user intent.
- The router accepts preferences defined in natural language or decision trees.
Technical Details:
- Built on a custom Mixture of Experts (MoE) model with <200ms latency and zero extra cost.
- Evaluation scores 90% correctness vs 95% for a single premium model, using fewer tokens and returning faster.
- The proxy and router are open-sourced, requiring no code changes.
More from coding & agent
- Perplexity open-sources SPACE, the production sandbox platform behind Perplexity Computer — AravSrinivas · 2026-10-16
- Meta releases Muse Spark Open: 816B MoE, 42B active, 1M context, day-0 on Baseten — baseten · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16
- Vercel makes Python generally available: FastAPI, Django, Fluid Compute support — steren · 2026-10-16
- Full course: build a playable Unity fishing game with Claude Opus 5.5 in six steps — Roger_M_Taylor · 2026-10-16
- Marc Lou praises PostShiba, a cute indie alternative to AWS SES for email infra — marclou · 2026-10-16