Millwright launches as a Rust self-hosted LLM router for agents and cost control
AndrewLiu96 · hn · 2026-07-23
Show HN: Millwright is a Rust-based, self-hosted LLM router aimed at cost savings, transparency, and performance for GenAI workloads and coding agents.
- Built as an open-source alternative to hosted routers like OpenRouter, Ramp Router, and Vercel AI Gateway.
- Supports OpenAI-compatible APIs, Anthropic, and Bedrock, plus Chat Completions and Anthropic Messages translation.
- Routing features include policy-based model roles, cheapest healthy route selection, cache affinity, spend tracking, and request tracing.
- Reliability features include bounded failover, circuit breakers, timeouts, and concurrency limits.
- Deployment is intentionally simple: one Rust binary, with Docker and SQLite/PostgreSQL support.
- The author says parts of the codebase were built with AI coding agents and is asking for feedback on routing policy, provider coverage, and self-hosting blockers.
More from coding & agent
- GPT-6 Astra beats Factorio with enemies in 44 in-game hours at ~$4,500 API cost — liminal_bardo · 2026-09-11
- 105 hidden bugs, 2 repos: DeepSeek V4.1 Flash fixes 24 at $1.80 vs Opus 5's 27 at $51.33 — ChartsJournalX · 2026-09-11
- Investment Analyst Asks How to Build a Claude-Based Diligence Agent Stack — Careless_Tie2286 · 2026-09-11
- Treating agents like 50 First Dates: a 3-layer context system so every conversation doesn't start from zero — evielync · 2026-09-11
- Running the Firefox MCP on Android via Termux, ngrok, and mcp-proxy — Nervous-Strain7544 · 2026-09-11
- SmolVM open-sources persistent computer infrastructure for agents that outlive chat sessions — aniketmaurya · 2026-09-11