Arize AX vs OpenLLMetry
Arize AX
arize.comOpenLLMetry
traceloop.comSide by side
| Figure | Arize AX | OpenLLMetry |
|---|---|---|
| Price from | $50/mo | — |
| Free plan | Yes | — |
| Free trial | — | — |
| Open source | — | — |
| Web | Yes | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 3 | 2 |
| Spec score · AI LLM Observability Tools | 8.4 (#9) | 6.9 (#23) |
| Spec score · LLM Observability Tools | 8.4 (#11) | 6.9 (#26) |
| Purpose | Arize AX is a platform for continuous improvement of AI agents that turns production behavior into better agents. | OpenLLMetry is an open source project for monitoring and debugging LLM app execution, with non-intrusive tracing built on OpenTelemetry. |
| Self-hosting | Self-hosted Arize AX keeps observability data, authentication, APIs and UI inside the customer’s infrastructure, with air-gapped deployment available. | Traceloop says it supports on-premise deployment in AWS, GCP, Azure, Kubernetes, air-gapped, and other restricted environments. |
| Support | AX Free includes community troubleshooting support, AX Pro includes email support and AX Enterprise includes dedicated support. | The Enterprise plan includes dedicated Slack support. |
Plans
In their own words
Arize AX
Arize AX is a platform for improving AI agents by turning production behavior into investigations, evaluations, and experiments. Signal surfaces important issues, helps identify causes, and prepares fixes for review. Managed Agents can investigate failing traces, fix repository bugs, automate recurring checks, or be created from scratch. Swarm Observability tracks status and performance across an agent swarm in real time. Agent-as-a-Judge uses telemetry to produce scalable evaluations that can surface unknown failures, while end-to-end experiments help validate changes. Integrations include OpenAI, Anthropic, Google GenAI, Amazon Bedrock, LangChain, LangGraph, LlamaIndex, CrewAI, and Vercel AI SDK; tracing follows OpenInference conventions and OpenTelemetry. AX Free costs nothing and includes 10 Signal issues/month, 25k trace spans/month, 1 GB ingestion/month, and 15-day retention. AX Pro costs $50 a month, with 25 Signal issues/month, 50k trace spans/month, 10 GB ingestion/month, and 30-day retention. Enterprise pricing is custom. Self-hosted deployment keeps observability data, authentication, APIs, and UI within customer infrastructure; air-gapped deployment is available.
OpenLLMetry
OpenLLMetry has no write-up on Specifiction yet; its figures are above.