DigitalOcean Open Sources Preference-Based Model Router
AI Engineer · youtube · 2026-08-22
Archana Kamath and Tyler Gillam present a model routing system based on preferences rather than benchmarks. Demos show session costs of 8 cents vs 25 cents for a single premium model, with the gap widening over time.
Core Concept:
- No single best model exists; the right choice depends on task, system prompt, tools, cost, latency, and user intent.
- The router accepts preferences defined in natural language or decision trees.
Technical Details:
- Built on a custom Mixture of Experts (MoE) model with <200ms latency and zero extra cost.
- Evaluation scores 90% correctness vs 95% for a single premium model, using fewer tokens and returning faster.
- The proxy and router are open-sourced, requiring no code changes.
More from coding & agent
- Agentic coding accessibility will reshape understanding of software complexity — pixlpa · 2026-08-24
- Devin Agent bypasses Slack block by finding emails in git logs — sandylikesfrogs · 2026-08-24
- Developer habits shift: Agents become collaborators from simple tools — latticecut · 2026-08-24
- Dev bottleneck shifts from writing to reading code: exe.dev co-founder — thursdai_pod · 2026-08-24
- The biggest AI mistake: trying to reinvent the wheel instead of using tools — Tired40s · 2026-08-24
- DeepPaperNote turns research papers into Obsidian notes — tom_doerr · 2026-08-24