Envoy AI Gateway vs LLM Gateway
Envoy AI Gateway
aigateway.envoyproxy.ioLLM Gateway
llmgateway.ioSide by side
| Figure | Envoy AI Gateway | LLM Gateway |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | Yes |
| Open source | — | Yes |
| Web | — | Yes |
| macOS | Yes | — |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 2 |
| Spec score · LLM Gateway Software | 8.4 (#8) | 8.7 (#6) |
| Spec score · LLM gateways | 8.4 (#10) | 9.1 (#3) |
| Observability | The CLI supports OpenTelemetry tracing and can record request and response details; the documentation describes configuration to hide input and output content. | Dashboards expose requests, tokens, latency, errors, cache status and spending broken down by model, provider, project and API key. |
| Security | The project describes upstream authentication for securing connections to AI services and says provider credentials can be kept out of application workloads. | The company states that it is SOC 2 Type II audited and GDPR-aligned, and describes TLS in transit and AES-256 encryption at rest. |
| Support | The project directs users to its community Discord and GitHub issues for help. | The Free plan lists Discord Community support, while Enterprise includes 24/7 priority support through a dedicated Slack or Discord channel. |
| Unified API | It provides an OpenAI-compatible API for routing requests across model providers and lets applications switch models by changing the model name. | Applications can target https://api.llmgateway.io/v1 and change model identifiers instead of rewriting provider integrations. |
Plans
In their own words
Envoy AI Gateway
Envoy AI Gateway, now named Agent Router, is an open-source control plane for managing AI and agent traffic through Envoy. It provides an OpenAI-compatible API for hosted and self-hosted models and MCP servers, with integrations for providers including OpenAI, Azure OpenAI, Google Gemini, AWS Bedrock, Mistral, and Anthropic. Traffic controls include model virtualization, provider failover, token quotas, and usage-based rate limiting. Its MCP gateway supports server multiplexing, tool routing, OAuth authentication, and observability; InferencePool support adds routing and load balancing for inference endpoints. Credentials can be managed centrally, and supported short-lived credential flows include AWS Bedrock, Azure OpenAI, and GCP Vertex AI. The project describes metrics collection and monitoring for AI workloads. The free open-source project is listed at free. Its CLI supports local use on Linux and macOS; production Kubernetes deployment uses Envoy Gateway and requires compatible dependencies. The project uses the Apache 2.0 license and points users to Discord and GitHub issues for help.
LLM Gateway
LLM Gateway provides one OpenAI-compatible API for routing requests among more than 250 models from over 40 providers. If a provider errors, times out, or rate-limits a request, it can retry up to twice with another provider for the same model. Its dashboard tracks request, token, spend, latency, error, and cache metrics, with model and provider breakdowns. Teams can use their own provider API keys without a platform fee and pay providers directly. The open-source core is licensed under AGPLv3 and can be self-hosted for free; enterprise features require a license. The Free plan is free, billed free forever, and has a 5% fee on credit usage. It includes 200+ models across 40+ providers, activity logs and export, and 20 requests per minute on free models; paid models are not rate limited. Enterprise pricing is custom and not listed. It includes custom routing, unlimited data retention, a 99.9% uptime SLA, and 24/7 priority support. Published integration guides cover tools including Claude Code, Cursor, Codex CLI, Cline, n8n, and VS Code.