Millwright opens as a self-hosted Rust router for mixed LLM providers
teachmehowtowookiee · reddit · 2026-07-28
A developer open-sourced Millwright, a self-hosted LLM router written in Rust and positioned as a transparent, cost-saving alternative to OpenRouter.
- It supports mixing self-hosted and hosted models in the same routing layer
- Provider support includes OpenAI-compatible APIs, Anthropic, and Amazon Bedrock
- Routing features include role-based model selection, cheapest-healthy-route picking, failover, circuit breakers, and concurrency limits
- It tracks spend per team, cache usage, model/provider mix, and request traces
- It can output cost analysis in HTML, Markdown, and JSON
- Deployment is packaged as a single Rust binary with Docker and SQLite/PostgreSQL support
More from coding & agent
- Panel on how AI agents will reshape software engineering features OpenAI, Google, Replit and Perplexity voices — steipete · 2026-07-28
- Open-source SongForge-MCP lets Claude generate full songs locally — RaVN3X · 2026-07-28
- Gemini CLI fixes macOS sandbox crashes by embedding Seatbelt profiles — amelidev · 2026-07-28
- Nativ v0.1.0 Released: Run AI Models Locally on Apple Silicon with Multi-IDE Support — max_paperclips · 2026-07-28
- CodeGraph made local token counts spike, but provider logs showed the run used fewer tokens — TruthTellerTom · 2026-07-28
- A practical guide to when to use ChatGPT, Claude, and agent modes — Simon Willison · 2026-07-28