Millwright launches as a Rust self-hosted LLM router for agents and cost control
AndrewLiu96 · hn · 2026-07-23
Show HN: Millwright is a Rust-based, self-hosted LLM router aimed at cost savings, transparency, and performance for GenAI workloads and coding agents.
- Built as an open-source alternative to hosted routers like OpenRouter, Ramp Router, and Vercel AI Gateway.
- Supports OpenAI-compatible APIs, Anthropic, and Bedrock, plus Chat Completions and Anthropic Messages translation.
- Routing features include policy-based model roles, cheapest healthy route selection, cache affinity, spend tracking, and request tracing.
- Reliability features include bounded failover, circuit breakers, timeouts, and concurrency limits.
- Deployment is intentionally simple: one Rust binary, with Docker and SQLite/PostgreSQL support.
- The author says parts of the codebase were built with AI coding agents and is asking for feedback on routing policy, provider coverage, and self-hosting blockers.
More from coding & agent
- Alex Townsend posts 200 open problems in numerical linear algebra for humans and AI agents — IgorCarron · 2026-09-11
- Kimi K2.8 Preview rolls out: near-K3 coding performance, 1M context for all tiers — teortaxesTex · 2026-09-11
- Looking for a classifier of software engineering task shapes to pick models per task — StewartalsopIII · 2026-09-11
- Steal this idea: prompt-to-hardware where agents assemble custom devices — paraschopra · 2026-09-11
- Model Is the Least Interesting Part: A Guide to Six Core AI Architectures from RAG to Multi-Agent — goyalshaliniuk · 2026-09-11
- Non-coder builds layered memory architecture: 20k tokens tracks a year of agent conversations — matteoianni · 2026-09-11