Skip to content
SPECIFICTION
Get listed

Envoy AI Gateway vs LiteLLM

Envoy AI Gateway

aigateway.envoyproxy.io

LiteLLM

litellm.ai

Side by side

FigureEnvoy AI GatewayLiteLLM
Price fromFreeFree
Free planYesYes
Free trial—Yes
Open source——
Web—Yes
macOSYes—
LinuxYes—
APIYesYes
Self-hostedYesYes
Plans published12
Spec score · LLM Gateway Software8.4 (#8)7.8 (#15)
Spec score · LLM gateways8.4 (#10)7.8 (#19)
Spec score · Byok System8.4 (#8)7.7 (#16)
DeploymentThe CLI can run locally on Linux and macOS, and production deployment uses Kubernetes with Envoy Gateway.The gateway can be self-hosted, including in air-gapped environments, and supports deployment with an official Helm chart or Terraform module.
Intended usersThe documentation identifies teams sharing models or tools, seeking centralized credentials, quotas, usage attribution, or failover as likely users.The Enterprise plan is described for teams running LiteLLM in production, including high-volume and regulated deployments.
SecurityThe project describes upstream authentication for securing connections to AI services and says provider credentials can be kept out of application workloads.The product page describes signed hardened non-root images, vulnerability scanning, SSO/JWT/RBAC controls, and secrets-manager integrations.
SupportThe project directs users to its community Discord and GitHub issues for help.Enterprise includes dedicated support and onboarding, with 24/7 support and SLAs listed on the pricing page.

Plans

Envoy AI Gateway

  • Open-source projectFree

Envoy AI Gateway pricing →

LiteLLM

  • Open SourceFree
  • EnterprisePriced on request

LiteLLM pricing →

In their own words

Envoy AI Gateway

Envoy AI Gateway, now named Agent Router, is an open-source control plane for managing AI and agent traffic through Envoy. It provides an OpenAI-compatible API for hosted and self-hosted models and MCP servers, with integrations for providers including OpenAI, Azure OpenAI, Google Gemini, AWS Bedrock, Mistral, and Anthropic. Traffic controls include model virtualization, provider failover, token quotas, and usage-based rate limiting. Its MCP gateway supports server multiplexing, tool routing, OAuth authentication, and observability; InferencePool support adds routing and load balancing for inference endpoints. Credentials can be managed centrally, and supported short-lived credential flows include AWS Bedrock, Azure OpenAI, and GCP Vertex AI. The project describes metrics collection and monitoring for AI workloads. The free open-source project is listed at free. Its CLI supports local use on Linux and macOS; production Kubernetes deployment uses Envoy Gateway and requires compatible dependencies. The project uses the Apache 2.0 license and points users to Discord and GitHub issues for help.

Visit aigateway.envoyproxy.io

LiteLLM

LiteLLM is an open-source AI gateway and LLM proxy for platform teams. It provides one OpenAI-compatible API to 140+ providers and 1,800+ models, with load balancing, lowest-cost routing and automatic selection of cheaper models for simple prompts. Teams can track usage and spend across keys, users, teams, organizations, tools, agents and MCP, then apply hard budgets and rate limits. Response and semantic caching can use Redis, S3 or GCS. LiteLLM can be self-hosted, including in air-gapped environments, and offers deployment through an official Helm chart or Terraform module. The maker says self-hosting has no telemetry and keeps data within the user's infrastructure. Listed security features include hardened non-root images, vulnerability scanning, SSO/JWT/RBAC controls and secrets-manager integrations. The free Open Source plan is self-hosted and includes 100+ providers, virtual keys, budgets, rate limits, fallbacks, logging and Prometheus metrics. A 30-day Enterprise trial key is available without a credit card.

Visit litellm.ai