LLM Gateway vs TrueFoundry AI Gateway
LLM Gateway
llmgateway.ioTrueFoundry AI Gateway
truefoundry.comSide by side
| Figure | LLM Gateway | TrueFoundry AI Gateway |
|---|---|---|
| Price from | Free | $25/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | Yes | — |
| Web | Yes | Yes |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 2 | 9 |
| Spec score · LLM gateways | 9.1 (#3) | 8.9 (#6) |
| Spec score · LLM Gateway Software | 8.7 (#6) | 8.7 (#7) |
| API surface | Documented endpoints include chat completions, responses, Anthropic messages, embeddings, images, speech, transcription, moderation, search, video and realtime operations. | The documentation exposes endpoints for chat, embeddings, reranking, responses, images, audio, batches, files, fine-tuning, moderation and model listing. |
| Documentation depth | The documentation includes feature guides, dashboard knowledge-base articles, integration material, self-hosting sections, developer tools and an API reference. | The docs include provider catalogs, authentication, self-hosted model setup, metrics APIs and a large endpoint reference covering multiple media and model operations. |
| Guardrails | Enterprise guardrails can detect prompt injection, jailbreak attempts, personally identifying information, payment-card data and secrets, with block, redact or warn actions. | Configurable input and output controls can filter PII, detect prompt injection, assess toxicity and apply organization-specific safety policies. |
| Observability | Dashboards expose requests, tokens, latency, errors, cache status and spending broken down by model, provider, project and API key. | Monitoring covers requests, tokens, cost, latency percentiles, time to first token, inter-token latency and request-level traces. |
| Routing | The gateway can retry up to twice with another provider for the same model when a provider errors, times out, or rate-limits a request. | The gateway supports latency-based routing, weighted load balancing and automatic fallbacks to secondary models. |
| Support | The Free plan lists Discord Community support, while Enterprise includes 24/7 priority support through a dedicated Slack or Discord channel. | The listed support options are an in-app chatbot and Discord for Developer, a support portal for Pro, and priority support, SLA and a dedicated Slack channel for Enterprise. |
| Unified API | Applications can target https://api.llmgateway.io/v1 and change model identifiers instead of rewriting provider integrations. | The gateway presents one API and key pattern across OpenAI, Anthropic, Bedrock, self-hosted models and other providers. |
| What it does | LLM Gateway places one OpenAI-compatible API between applications and dozens of model providers, handling routing, credentials, usage and cost tracking. | TrueFoundry AI Gateway is a control layer that connects applications to models, tools and agents while centralizing routing, governance, observability and cost controls. |
Plans
TrueFoundry AI Gateway
- DeveloperFree
- DeveloperFree
- Pro$25 a month
- Pro$499 a month
- Pro Plus$2,999 a month
- TrueFoundry Pro (AWS Marketplace)$6,250 a month
- TrueFoundry Enterprise (AWS Marketplace)$12,500 a month
- TrueFoundry On-Prem (AWS Marketplace)$16,666.66 a month
- EnterprisePriced on request
In their own words
LLM Gateway
LLM Gateway provides one OpenAI-compatible API for routing requests among more than 250 models from over 40 providers. If a provider errors, times out, or rate-limits a request, it can retry up to twice with another provider for the same model. Its dashboard tracks request, token, spend, latency, error, and cache metrics, with model and provider breakdowns. Teams can use their own provider API keys without a platform fee and pay providers directly. The open-source core is licensed under AGPLv3 and can be self-hosted for free; enterprise features require a license. The Free plan is free, billed free forever, and has a 5% fee on credit usage. It includes 200+ models across 40+ providers, activity logs and export, and 20 requests per minute on free models; paid models are not rate limited. Enterprise pricing is custom and not listed. It includes custom routing, unlimited data retention, a 99.9% uptime SLA, and 24/7 priority support. Published integration guides cover tools including Claude Code, Cursor, Codex CLI, Cline, n8n, and VS Code.
TrueFoundry AI Gateway
TrueFoundry AI Gateway sits between applications and model providers to manage model access, routing, governance, observability, security, and cost controls. Its unified API supports chat, completion, embedding, and reranking models. TrueFoundry says the gateway connects to more than 1,000 LLMs across more than 35 platforms, including OpenAI, AWS Bedrock, Google Vertex AI, Azure OpenAI, and Anthropic. Routing options include latency-based selection, weighted load balancing, and automatic fallbacks. The service tracks token usage, latency, errors, and request volumes, and supports centrally stored request and response logs. Teams can set rate limits and cost or token quotas, with role-based access controls. MCP connections include tools such as Slack, GitHub, Confluence, and Datadog, with policies for tool calls. Deployment includes SaaS, with VPC, on-premises, and air-gapped options for Enterprise. Developer is free; Pro costs $25 a month, billed per user. Developer includes 10,000 requests per user monthly, while Pro includes 20,000 and charges $20 per additional 100,000 requests.