Arize Phoenix vs Portkey

Arize Phoenix is free, open-source software for tracing, evaluating, debugging, and improving AI applications. It captures model calls, retrieval, tool use, and custom logic as OpenTelemetry traces. Teams can score traces and spans with LLM evaluators, code checks, or human labels. Prompt tools provide versioning, side-by-side testing, and call replay, while dataset experiments compare evaluation results across application versions. Documented integrations include LlamaIndex, LangChain, DSPy, OpenAI, Bedrock, and Anthropic, plus evaluation libraries such as Harbor, Ragas, Cleanlab, and MLflow. Phoenix runs locally or can be self-hosted with Docker, Kubernetes, or a cloud environment; Arize says self-hosting keeps traces in the user’s environment. Self-hosted authentication supports OAuth2 identity providers with an OpenID Connect well-known endpoint. The Phoenix plan costs nothing and leaves trace spans, ingestion, projects, and retention under user management. The project is ELv2 licensed and built by Arize AI and the open-source community.

Portkey is a platform for managing, observing, and governing generative AI across teams. Its gateway offers a unified API for models and providers across modalities, with configurable routing, failovers, retries, and load balancing. Monitoring tools include logs, traces, feedback, metadata, filters, and alerts; prompt management adds templates, a playground, API endpoints, versioning, and variables. Listed integrations include OpenAI, Anthropic, Google Gemini, Azure, Amazon Bedrock, and Ollama, as well as agent frameworks including CrewAI, LangChain, LangGraph, and the OpenAI Agents SDK. The gateway offers simple and semantic caching for repeat requests. Security features include virtual provider keys, access controls, inbound IP and geographic rules, and sensitive-data redaction. Self-hosted and local deployment options are listed, while Enterprise adds private cloud deployment and VPC hosting. Developer is free and includes 10k recorded logs/month, 3 days log retention, and 30 days metrics retention. Production costs $49 a month, billed $49/month, with 100k recorded logs/month and production support. Enterprise pricing is custom. Portkey lists San Francisco as its headquarters and March 2023 as its launch date.