OpenZiti LLM Gateway vs TensorZero
OpenZiti LLM Gateway
github.com
TensorZero
tensorzero.comSide by side
| Figure | OpenZiti LLM Gateway | TensorZero |
|---|---|---|
| Price from | — | Free |
| Free plan | — | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 0 | 1 |
| Spec score · LLM gateways | 7.1 (#27) | 8.6 (#7) |
| Spec score · LLM Gateway Software | 7.4 (#28) | 8.0 (#13) |
| Deployment | The project describes deployment as one Go binary and one YAML file, with no database, message queue, or sidecar required. | The gateway can be deployed with Docker, Docker Compose, Kubernetes using a reference Helm chart, or built from source with Cargo. |
| Integrations | Documented backends include OpenAI, Anthropic, Ollama, vLLM, llama-server, and SGLang; clients use an OpenAI-compatible API. | Documented providers include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, and OpenAI-compatible endpoints. |
| License | The repository identifies the project as Apache 2.0 licensed. | The FAQ describes TensorZero as open source under the Apache 2.0 License. |
| Observability | When enabled, Prometheus-exported metrics include request counts, latency, token counts, routing decisions, provider errors, and endpoint health. | TensorZero can store inference and feedback data for observability, optimization, evaluations, and experimentation in Postgres or ClickHouse. |
Plans
In their own words
OpenZiti LLM Gateway
OpenZiti LLM Gateway has no write-up on Specifiction yet; its figures are above.
TensorZero
TensorZero is a free, open-source stack that brings together an LLM gateway, observability, optimization, evaluation, and experimentation. Its gateway provides a unified API for model providers and supports routing, retries, fallbacks, prompt templates, schemas, multi-step workflows, and traffic routing for A/B tests. Documented integrations include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, xAI, and OpenAI-compatible endpoints. Inference and feedback data go to the database you configure, where they can be queried with SQL; feedback can be tied to individual inferences or episodes. The gateway and UI can run through Docker, Docker Compose, Kubernetes and Helm, or a source build. API key authentication is available for gateway endpoints other than status and health checks. The project is available on GitHub, but is no longer maintained; its repository is archived and read-only. TensorZero is self-hosted and licensed under Apache 2.0. It collects pseudonymous aggregate usage analytics, excluding application data and inference inputs and outputs, with an option to disable collection.