Open-source auto-router prices every model per request, and shows warm cache beats cheaper models
airesearch12 · x · 2026-09-19
A developer replaced the classifier LLM in his auto-router with a small model, making routing nearly instant and free, and released an open-source playground. Each request is classified in 0.5s; every candidate model is priced for that specific request including redo risk, and the cheapest expected total wins. The key insight: a warm provider cache bills the next turn at 1/10 input price, so the real question each turn is whether a better model is worth abandoning the warm cache. The playground exposes per-route pricing, measured success rates, and expected costs, and can be run locally.
More from coding & agent
- Meta Muse hands-on: fails as tasks pile up, "scales to millions but not personally" — deliprao · 2026-09-19
- Scribble handwriting app for kids built by 2 people + AI agents with 366 PRs opens first cohort — LamarDealMaker · 2026-09-19
- Open-source Nautilo adds namespaced memory so AI agents can't leak secrets across rooms — Dan_Jeffries1 · 2026-09-19
- Berkeley's PixelRAG hits 10k stars: screenshots beat text parsing for RAG retrieval — tom_doerr · 2026-09-19
- Dev builds own AI video transcription platform, saves $120/year on SaaS — AssemblyAI · 2026-09-19
- Agent handles entire expense report: mines iMessage, pulls Gmail receipts — cameronstow · 2026-09-19