LiteLLM

LiteLLM is an open-source AI gateway and proxy that gives developers one OpenAI-compatible interface for calling many model providers. It helps platform teams manage model access, routing, fallbacks, budgets, rate limits, prompt management, pass-through endpoints, logging, and spend tracking across providers such as OpenAI, Azure, Anthropic, Gemini, Bedrock, and others. The product is especially useful when an AI product needs provider flexibility instead of being locked to one API. Teams can route requests through LiteLLM, standardize model names, attribute usage to users or teams, set spend controls, and add observability around production LLM traffic. LiteLLM has an open-source core and commercial enterprise gateway options. The public site emphasizes on-prem deployment, model access, budgets, rate limiting, guardrails, cost tracking, and enterprise requests rather than simple self-serve plan cards. For FutureStack, LiteLLM belongs in Developer Tools because it solves a real infrastructure problem for AI builders. It is best for teams that need a reliable LLM gateway, multi-provider routing, cost controls, fallbacks, and production model governance.