LangSmith vs LLM Gateway
LangSmith
langchain.comLLM Gateway
llmgateway.ioSide by side
| Figure | LangSmith | LLM Gateway |
|---|---|---|
| Price from | $39/mo | Free |
| Free plan | Yes | Yes |
| Free trial | — | Yes |
| Open source | — | Yes |
| Web | Yes | Yes |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 3 | 2 |
| Spec score · LLM Gateway Software | 9.2 (#1) | 8.7 (#6) |
| Spec score · LLM gateways | 9.3 (#2) | 9.1 (#3) |
| Integrations | Documented integrations cover OpenAI, Anthropic, Bedrock, Gemini, Mistral, LiteLLM, OpenTelemetry, CrewAI, Vercel AI SDK, PydanticAI, AutoGen and many other frameworks and providers. | Published integration guides cover Claude Code, Cursor, Cline, n8n, VS Code, Codex CLI, OpenCode, Continue, Kilo Code, Kimi Code and other coding tools. |
| Observability | Its observability features include agent tracing, cost tracking, online LLM-as-judge and code evaluations, trajectory monitoring, and webhook or PagerDuty alerts. | Dashboards expose requests, tokens, latency, errors, cache status and spending broken down by model, provider, project and API key. |
| Security | LangSmith's trust center lists SOC 2 Type II, ISO 27001:2022, GDPR and HIPAA compliance. | The company states that it is SOC 2 Type II audited and GDPR-aligned, and describes TLS in transit and AES-256 encryption at rest. |
| Self-hosting | Enterprise customers can run LangSmith self-hosted or hybrid on Kubernetes in AWS, Google Cloud or Azure. | The gateway can run on customer infrastructure using Docker or Kubernetes with a Helm chart; Enterprise features require a license key. |
| Support | The Developer plan includes community support, and the Enterprise plan lists a support SLA. | The Free plan lists Discord Community support, while Enterprise includes 24/7 priority support through a dedicated Slack or Discord channel. |
| What it does | LangSmith is LangChain's framework-agnostic platform for observing, evaluating and deploying AI agents. | LLM Gateway places one OpenAI-compatible API between applications and dozens of model providers, handling routing, credentials, usage and cost tracking. |
Plans
In their own words
LangSmith
LangSmith is a commercial agent-engineering platform for building, evaluating, deploying, monitoring, and improving AI agents. It supports offline evaluations on curated datasets and real-time scoring of production interactions. Evaluation methods include human review, heuristic checks, LLM-as-judge evaluators, pairwise comparisons, and custom Python or TypeScript evaluators. Annotation queues route runs to subject-matter experts for feedback that can calibrate evaluations, improve prompts, or augment datasets. The Playground compares prompt versions or model providers, and LangSmith chat can auto-improve prompts. Observability includes agent and retrieval tracing, token and general cost tracking, trajectory monitoring, online evaluations, and webhook or PagerDuty alerts. Traces can arrive through SDKs or an API from LangGraph, custom Python, or other stacks; SDKs are listed for Python, TypeScript, Go, and Java. Hosting ranges from cloud to hybrid and self-hosted Enterprise options. The Developer plan costs nothing, Plus costs $39 a month, and Enterprise pricing is custom.
LLM Gateway
LLM Gateway provides one OpenAI-compatible API for routing requests among more than 250 models from over 40 providers. If a provider errors, times out, or rate-limits a request, it can retry up to twice with another provider for the same model. Its dashboard tracks request, token, spend, latency, error, and cache metrics, with model and provider breakdowns. Teams can use their own provider API keys without a platform fee and pay providers directly. The open-source core is licensed under AGPLv3 and can be self-hosted for free; enterprise features require a license. The Free plan is free, billed free forever, and has a 5% fee on credit usage. It includes 200+ models across 40+ providers, activity logs and export, and 20 requests per minute on free models; paid models are not rate limited. Enterprise pricing is custom and not listed. It includes custom routing, unlimited data retention, a 99.9% uptime SLA, and 24/7 priority support. Published integration guides cover tools including Claude Code, Cursor, Codex CLI, Cline, n8n, and VS Code.