TensorZero
tensorzero.com
What it is
TensorZero is a free, open-source stack that brings together an LLM gateway, observability, optimization, evaluation, and experimentation. Its gateway provides a unified API for model providers and supports routing, retries, fallbacks, prompt templates, schemas, multi-step workflows, and traffic routing for A/B tests. Documented integrations include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, xAI, and OpenAI-compatible endpoints. Inference and feedback data go to the database you configure, where they can be queried with SQL; feedback can be tied to individual inferences or episodes. The gateway and UI can run through Docker, Docker Compose, Kubernetes and Helm, or a source build. API key authentication is available for gateway endpoints other than status and health checks. The project is available on GitHub, but is no longer maintained; its repository is archived and read-only. TensorZero is self-hosted and licensed under Apache 2.0. It collects pseudonymous aggregate usage analytics, excluding application data and inference inputs and outputs, with an option to disable collection.
Spec sheet
- Product
- TensorZero is an open-source platform that combines an LLM gateway, observability, evaluation, optimization, and experimentation.Read 7 Oct 2026 · github.com
- Deployment
- The gateway can be deployed with Docker, Docker Compose, Kubernetes using a reference Helm chart, or built from source with Cargo.Read 7 Oct 2026 · github.com
- Integrations
- Documented providers include Anthropic, AWS Bedrock, Azure, Google, OpenAI, OpenRouter, Together AI, vLLM, and OpenAI-compatible endpoints.Read 7 Oct 2026 · tensorzero.com
- Security
- TensorZero's privacy policy says it uses physical, technical, organizational, and administrative security measures to protect personal data.Read 7 Oct 2026 · tensorzero.com
- Privacy
- The gateway documentation says its pseudonymous usage analytics do not include application data or inference inputs and outputs, and can be disabled.Read 7 Oct 2026 · github.com
- Support
- The project README directs users to Slack or Discord and offers work teams a Slack or Teams channel by email.Read 7 Oct 2026 · github.com
- License
- The FAQ describes TensorZero as open source under the Apache 2.0 License.Read 7 Oct 2026 · tensorzero.com
- Company
- The FAQ describes the team as based in New York City and names Viraj Mehta and Gabriel Bianconi as its founders.Read 7 Oct 2026 · tensorzero.com
- Clients
- TensorZero supports the OpenAI SDK, OpenTelemetry, and clients for major programming languages.Read 7 Oct 2026 · github.com
- Current status
- The homepage says TensorZero remains available on GitHub but is no longer maintained.Read 7 Oct 2026 · tensorzero.com
- Data control
- TensorZero documentation says it stores its data in the user's own database, which can be queried directly with SQL.Read 7 Oct 2026 · tensorzero.com
- Experiments and workflows
- The gateway supports multi-step workflows and traffic routing for A/B tests, with feedback assignable to individual inferences or episodes.Read 7 Oct 2026 · tensorzero.com
- Features
- The product includes routing, retries, fallbacks, prompt templates, schemas, and A/B testing.Read 7 Oct 2026 · github.com
- Gateway
- The gateway provides a unified API for accessing supported LLM providers.Read 7 Oct 2026 · tensorzero.com
- Maintenance status
- The official site says TensorZero remains available on GitHub but is no longer maintained.Read 7 Oct 2026 · tensorzero.com
- Model integrations
- Documented providers include Anthropic, AWS Bedrock, Azure, Google AI Studio, OpenAI, OpenRouter, vLLM, and xAI; OpenAI-compatible APIs are also supported.Read 7 Oct 2026 · tensorzero.com
- Notable limitation
- The FAQ says TensorZero had no revenue model at the time described and planned a managed service for the future; the current homepage says the project is no longer maintained.Read 7 Oct 2026 · tensorzero.com
- Observability
- TensorZero can store inference and feedback data for observability, optimization, evaluations, and experimentation in Postgres or ClickHouse.Read 7 Oct 2026 · github.com
- Project status
- GitHub marks the TensorZero repository as archived and read-only, archived on June 12, 2026.Read 7 Oct 2026 · github.com
- Usage analytics
- The gateway collects pseudonymous aggregate usage analytics, says these exclude application data, and provides a setting to disable collection.Read 7 Oct 2026 · tensorzero.com
Plans and prices
TensorZero LLMOps platform Free 100% self-hosted · open-source Read 7 Oct 2026 · github.com
Strong points and limits
Strong points
- Free, open-source stack under Apache 2.0
- Supports routing, retries, and fallbacks
- Inference data stays in the configured database
- Deployable with Docker or Kubernetes and Helm
- Usage analytics can be disabled
Limits
- Project is no longer maintained
- GitHub repository is archived and read-only
- Self-hosted deployment is required
- Usage analytics are collected by default
Who it suits
TensorZero may suit developers who want a self-hosted LLM gateway and data stored in their own database. Its archived, unmaintained status is important for anyone considering it for ongoing use.
Verdict
TensorZero offers a broad set of LLM gateway and experimentation capabilities, with data held in a user-configured database. Its no-longer-maintained status and archived repository are significant considerations.
Where Specifiction places it
- LLM gateways#7 of 338.6
- LLM Gateway Software#13 of 348.0
- Byok System#15 of 247.9
Alternatives to TensorZeroAll alternatives →
LangSmith From $39/mo · Free plan 9.3 Visit 
Bifrost Free plan 9.0 Visit