LiteLLM
LiteLLM is the default pick for a platform team that wants one OpenAI-compatible door to many model vendors, with per-team budgets and keys, without paying a hosted router a margin. The SDK is just as useful to a single developer who wants to swap providers with a one-line change. Running the gateway reliably, including its database and scaling, is your job, and the governance features that large companies ask for sit behind the Enterprise licence.
Pros
- One OpenAI-style interface to 100+ providers, in an SDK and a self-hosted proxy
- Virtual keys, spend tracking by key, user and team, budgets and rate limits in the free gateway
- Retry and fallback routing across deployments, plus MCP and agent gateways
- Self-hosted, so traffic and keys stay in your own environment
Cons
- You run and scale the proxy yourself, including its database and upgrades
- SSO, SCIM, audit logs and SLAs require the Enterprise licence, priced only by quote
- The unified format can lag behind provider-specific features on new models