Dev builds Jev-based auto LLM router with live playground, 679ms decisions
airesearch12 · x · 2026-09-18
A developer built a Jev-based Auto-LLM-Router with a live playground; each routing decision takes 679ms. The router classifies each request, prices every candidate model at what it actually costs including redo risk, and picks the cheapest expected total.
The key insight: a warm cache can be worth more than a cheaper model. A provider that has already read your conversation bills the next turn at roughly a tenth of the input price, while switching models rewrites the whole prefix at full price — so the real question each turn is "is the better model worth leaving a warm cache for," and the answer changes as the cache expires. The playground also lets you query the policy object directly to see how a 100k-token agent prefix affects routing decisions without paying for a call.
Related event: Developers Build Jev-Based LLM Router: Faster and Cheaper Routing(3 posts)→
More from coding & agent
- Install Jev with one npx command and let it replace slow, expensive LLM calls — iannuttall · 2026-09-19
- obsidian-second-brain: 4.5k-star open-source persistent memory for Claude Code via Obsidian — tom_doerr · 2026-09-19
- André is building a way to never touch GitHub Actions YAML again: Workflows beyond CI/CD — dinasaur_404 · 2026-09-19
- Dev predicts one model generation until agents beat bespoke workflows on cost — brandon_galang · 2026-09-19
- Pairing Codex with Jev for browser automation: 5-10x speedup with a split-labor workflow — alexcovo_eth · 2026-09-19
- 'AI Can Do Everything Except Go to Jail for You': Quality Rules for Agent Coding — dotey · 2026-09-19