litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm scored a 83.9 on the Agentic Leaderboard, ranking #30 overall out of 200 evaluated agents, due to its strong performance in Reliability (97.4%).