Overview

The gateway exposes one OpenAI-compatible API for more than 250 models from over 40 providers. That single integration point reduces the need to rework applications when changing providers or models. If a provider errors, times out, or rate-limits a request, the gateway can retry up to twice with another provider serving the same model. This is a practical availability safeguard, although teams still choose providers and remain responsible for their provider accounts.

Requests can be organized through virtual API keys, budget controls, and fallback routing. The dashboard reports request, token, spend, latency, error, and cache metrics, with views by model and provider. Those dimensions are useful for finding expensive or unreliable routes rather than treating all traffic as one undifferentiated total.

Bring-your-own-key operation has no platform fee: the provider bills the user directly. Credit-based use is different, with the Free plan charging 5% on credit usage. Metadata retention is free, while full payload retention costs $0.01 per 1 million tokens; Free includes activity logs and export.

Key features

Routing and operations

Two retry attempts with another provider give latency- or outage-sensitive applications a defined fallback behavior. The gateway's coverage of 250+ models and 40+ providers is broad enough for teams standardizing several integrations behind one endpoint. Custom routing is an Enterprise feature, so the Free plan is better for straightforward provider failover than for elaborate traffic policies.

Analytics and access

Breakdowns for spend, tokens, latency, errors, requests, and cache activity help teams allocate costs and investigate service quality. Virtual API keys and budget controls support a shared gateway for multiple applications or teams. They provide governance primitives, but the published plans do not promise automatic optimization of provider prices or model choice.

Deployment and security

The core gateway is open source under AGPLv3 and can be self-hosted at no cost. Enterprise features require a license. The company describes SOC 2 Type II auditing, GDPR alignment, TLS in transit, and AES-256 encryption at rest. Enterprise guardrails can detect prompt injection, personally identifiable information, and secrets, with central configuration. Enterprise SSO supports SAML 2.0 and OIDC, plus SCIM provisioning and group-based role mapping. Those controls are relevant to organizations deploying LLM infrastructure across departments, but they are not part of the free offering.

Published integration guides cover Claude Code, Cursor, Codex CLI, Cline, Continue CLI, GitHub Copilot, n8n, OpenCode, and VS Code, giving teams using those tools a documented route into the gateway.

Pricing

Free

The Free plan is free forever. It covers 200+ models across 40+ providers, allows 20 requests per minute on free models, and does not rate-limit paid models. Credit usage carries a 5% fee; BYOK avoids a platform fee but leaves provider charges with the user. Discord Community support, activity logs, and export are included. This is a sensible starting point for prototypes and smaller deployments that can stay within the free-model request quota.

Enterprise

Enterprise uses custom pricing and requires contacting the company. It adds volume discounts, custom routing, unlimited data retention, a 99.9% uptime SLA, and 24/7 priority support through a dedicated Slack or Discord channel. Enterprise plans include unlimited seats. Organizations needing SSO, SCIM, centrally managed guardrails, or contractual availability should budget for this tier; it is not a small step-up from Free but a separately negotiated deployment.

Platforms

LLM Gateway is offered as an API and web service, runs on Linux, and supports self-hosted deployment. The self-hosted route is particularly relevant to teams that need control over where the AGPLv3 core runs, while web access suits teams that prefer the hosted dashboard.

Who it's for

Product and platform teams are the primary fit: one API, provider fallback, virtual keys, budgets, and analytics address the operational friction of running several model integrations. Self-hosting makes it attractive to teams able to operate open-source infrastructure and wanting to avoid a platform fee when using their own keys. Enterprise buyers should consider it when multiple teams need centralized identity, retention, guardrails, support, and an uptime commitment. It is less compelling for a small project that only needs one provider and has no need for routing or shared usage controls.

Pros and cons

Pros

Broad provider coverage: one OpenAI-compatible endpoint spans 250+ models and 40+ providers.
Defined failover: up to two retries with another provider can reduce disruption from errors, timeouts, and rate limits.
Useful cost visibility: model- and provider-level spend, token, latency, error, and cache metrics support practical oversight.
Flexible economics: BYOK has no platform fee, and the AGPLv3 core can be self-hosted for free.
Serious enterprise controls: SSO, SCIM, guardrails, unlimited seats and retention, SLA, and priority support are available together.

Cons

Free request ceiling: free models are limited to 20 requests per minute, which can constrain busy prototypes.
Credit surcharge: the Free plan's 5% charge makes BYOK more attractive for cost-sensitive users.
Enterprise dependency: custom routing, identity controls, guardrails, and contractual support require custom pricing.
Self-hosting responsibility: running the open-source core shifts deployment and operations work to the customer.

Alternatives

Choose Bifrost instead if an Apache 2.0 self-hosted gateway and deployment through npx, Docker, Kubernetes, or a Go binary are the priority.

Kong Gateway is the better fit when a 30-day, no-credit-card trial and a broader API-gateway product are more important than LLM Gateway's focused routing model.

Pick YoloRouter for a self-hosted single binary with SQLite out of the box and optional PostgreSQL.

Requesty suits teams wanting a free allowance of 200 requests per day, routing, caching, fallbacks, spend analytics, and EU data residency.

Choose TrueFoundry AI Gateway when a per-user developer plan and explicit user, request, model-account, and MCP-server quotas match your rollout.

WSO2 AI Gateway is an alternative for organizations evaluating a managed 30-day trial with stated API, MCP, event, and hybrid-gateway allowances.

Envoy AI Gateway is a free open-source project for self-managed Linux or macOS deployments.

Consider Helicone when an open-source observability-focused gateway with a free Hobby allowance is the preferred starting point.

Verdict

LLM Gateway is a strong choice for product and platform teams that want one API, provider failover, and actionable usage accounting, with the option to bring their own keys or self-host the AGPLv3 core. Choose it especially when several applications share model infrastructure. Look elsewhere if you need enterprise controls without a custom contract, cannot operate a self-hosted service, or your workload is small enough that a single provider needs no gateway.

	Kong AI Gateway	LLM Gateway
Free plan	No	Yes
Free trial		Yes
Paid from		Free
Platforms	Web	api, Linux, self-hosted, Web
Free plan	No	Yes
Fallback routing	Yes	Yes
Usage analytics	Yes	Yes
Self-hosted deployment	Yes	Yes
Budget controls	Yes	Yes
Virtual API keys	Yes	Yes

Kong AI Gateway vs LLM Gateway

Kong AI Gateway

LLM Gateway