Millwright launches as a Rust self-hosted LLM router for agents and cost control
AndrewLiu96 · hn · 2026-07-23
Show HN: Millwright is a Rust-based, self-hosted LLM router aimed at cost savings, transparency, and performance for GenAI workloads and coding agents.
- Built as an open-source alternative to hosted routers like OpenRouter, Ramp Router, and Vercel AI Gateway.
- Supports OpenAI-compatible APIs, Anthropic, and Bedrock, plus Chat Completions and Anthropic Messages translation.
- Routing features include policy-based model roles, cheapest healthy route selection, cache affinity, spend tracking, and request tracing.
- Reliability features include bounded failover, circuit breakers, timeouts, and concurrency limits.
- Deployment is intentionally simple: one Rust binary, with Docker and SQLite/PostgreSQL support.
- The author says parts of the codebase were built with AI coding agents and is asking for feedback on routing policy, provider coverage, and self-hosting blockers.
More from coding & agent
- NVIDIA says Nemotron 3 Ultra hit 97.1% on agentic RTL chip-design tasks — NVIDIAAI · 2026-07-27
- Tokyo Agent Forge hackathon shipped production-ready AI agents in one day — DavidBennett__ · 2026-07-27
- Long-running agents will need immutable event logs, this thread argues — sebpaquet · 2026-07-27
- Agentic Data Science in Practice: Agents Write Code but Answer Wrong Questions — hugobowne · 2026-07-27
- A VS Code extension adds Markdown-style highlighting to Alchemy string templates — samgoodwin89 · 2026-07-27
- Claude’s Stripe MCP connector is being called unusable after repeated disconnects — evielync · 2026-07-27