Millwright
A free, open-source tool you run yourself that sits between your app and AI providers like OpenAI or Anthropic, automatically picking the cheapest model that can do the job and switching providers if one goes down.
🔗 Visit Millwright
Description
If you're paying for GPT calls, Claude calls, and Bedrock calls across different parts of your app, you've probably felt the pain of juggling API keys, watching costs creep up, and scrambling when a provider has an outage. Millwright is a small program you run on your own server that sits in the middle: your app talks to Millwright, and Millwright decides which AI provider and model actually handles each request, based on rules you set for cost and quality.
Millwright is an open-source, self-hosted LLM router written in Rust that routes requests across OpenAI, Anthropic, and Amazon Bedrock based on configurable policies spanning cheap/mid/frontier model tiers. It maintains cache affinity per session (routing repeat requests from the same session to the same provider to preserve prompt-caching benefits), tracks spend via a built-in ledger backed by SQLite or PostgreSQL, and includes automatic failover with circuit breakers so a provider outage doesn't take your app down. It ships as a single Rust binary with CLI commands (spend, models, trace, top) for operating it. Apache-2.0 licensed and completely free; being self-hosted, your API keys and traffic never leave your own infrastructure. It's a small project (under 30 GitHub stars, 5 commits at time of review), maintained by an organization called Northwood-Systems that also offers enterprise deployment consulting per its README, so functional but early-stage.
💬 Our review
The short version: if you specifically want to keep AI provider credentials and routing logic on your own infrastructure rather than trusting a third-party gateway, Millwright does that job for free with real cost-tracking and failover features, but its tiny GitHub footprint means you're an early adopter, not making a safe default choice.
Against hosted routers like OpenRouter, Millwright's whole value proposition is self-hosting: your traffic and keys never touch a third party, which matters if you have compliance or trust concerns a hosted service can't satisfy. Against LiteLLM, the most established self-hosted router in this space, Millwright is dramatically newer and smaller (LiteLLM has years of production use and a large community; Millwright has 5 commits and under 30 stars), so LiteLLM is the safer bet today unless Millwright's specific cache-affinity and single-binary simplicity is a decisive advantage for you. Free either way, so the real cost of trying Millwright is your own setup and maintenance time, and the risk that a single-digit-commit project could stall.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Gratuit, licence Apache-2.0. Coûts = uniquement votre propre infrastructure d'hébergement + vos clés API fournisseurs. Consulting entreprise disponible séparément.
Pros
Auto-hébergé — clés API et trafic ne quittent jamais votre infra
Affinité de cache par session pour préserver le prompt caching
Failover automatique avec circuit breakers
Suivi des dépenses intégré (SQLite/PostgreSQL)
Cons
Projet très jeune : moins de 30 étoiles GitHub, 5 commits
Bien moins mature que LiteLLM
Nécessite de gérer soi-même l'hébergement
Pas de date de lancement publique