Switchyard
A free, open-source traffic router for AI apps that lets you send requests to different AI models based on rules (cost, load, task type) while keeping the same OpenAI/Anthropic-style API your code already uses.
🔗 Visit Switchyard
Description
Apps that rely on a single AI provider are one outage or price hike away from a bad day. Switchyard is a router that sits between your app and multiple AI model providers, letting you switch, split or escalate traffic between them — cheap model for easy tasks, expensive one for hard tasks, automatic failover — without rewriting your application code each time.
Switchyard is an open-source (Apache 2.0) Rust proxy and library built by NVIDIA's NeMo team. It translates between OpenAI and Anthropic API formats so your existing client code keeps working regardless of which backend actually serves the request, and supports several routing strategies: classifier-based routing (an AI model decides which backend a request should go to), staged rollouts, escalation routing (retry on a stronger model if a weaker one fails), random routing for A/B testing, and simple passthrough. It ships Prometheus metrics for observability and can run as a standalone server or be embedded as a library.
💬 Our review
The short version: Switchyard is worth watching if you're building an LLM app and want provider flexibility baked in from day one, but its own maintainers label it pre-alpha, so treat it as an interesting building block rather than something to put straight into production.
The differentiator versus a general-purpose gateway is the routing sophistication — classifier-based and escalation routing in particular go beyond simple load-balancing, letting you route a request to a cheap model first and only escalate to an expensive one if needed, which directly targets AI API cost control. Being backed by NVIDIA's NeMo team is also a meaningfully different signal than a solo open-source project — it suggests real engineering investment and a plausible long-term roadmap, even if the project itself is young.
The honest limits: the project explicitly states it's pre-alpha with breaking API changes expected before a 1.0 release, so anyone adopting it now should expect to keep up with changes. It's also a Rust project, so deep customization assumes Rust familiarity even though most users will interact with it purely as a server/proxy. For a team that wants to experiment with multi-provider LLM routing and doesn't mind being an early adopter, it's worth a look; for a production system today, a more mature gateway is the safer choice until Switchyard stabilizes.
💰 Pricing
📊 Global score
🤖 AI-enriched data
Outil open-source (Apache 2.0), entièrement gratuit ; à héberger soi-même.
Pros
Routage sophistiqué (classificateur, escalade, A/B) au-delà du simple équilibrage de charge
Compatibilité native OpenAI/Anthropic — le code client existant continue de fonctionner
Soutenu par l'équipe NeMo de NVIDIA — investissement d'ingénierie sérieux
Gratuit, open-source (Apache 2.0), utilisable en serveur ou en bibliothèque Rust
Cons
Statut explicitement pré-alpha — changements d'API cassants attendus avant la v1.0
Pas prêt pour la production selon les mainteneurs eux-mêmes
Personnalisation profonde nécessite des connaissances Rust