LiteLLM Overview
LiteLLM Overview
Summary
LiteLLM is an open-source tool and proxy server that enables developers to call over 100 large language models (LLMs) from different providers using a unified OpenAI-compatible API. It supports both a Python SDK and an AI Gateway (proxy server) for flexible integration and deployment.
Content
What is LiteLLM?
LiteLLM allows users to interact with a wide range of LLMs (OpenAI, Bedrock, Azure, VertexAI, Anthropic, Groq, and more) through a single interface. It provides endpoints for chat, completions, embeddings, images, audio, and more, and supports agent invocation and advanced routing.
Key Features
- Unified API for 100+ LLMs
- Python SDK and AI Gateway (proxy server)
- Support for multiple endpoints and providers
- Agent invocation and A2A protocol support
- Easy deployment via Docker, Render, or Railway
Use Cases
- Developers building multi-provider LLM applications
- Organizations seeking to simplify LLM integration and management
- Teams needing a flexible, open-source AI gateway