Phoenix

Open-source LLM and agent observability platform with tracing, evals, and experimentation built on OpenTelemetry.

Freemium· Open source (ELv2) + free Phoenix Cloud; paid Arize AX for enterpriseEvaluationMulti-model

Best for

Pick Phoenix if you're building LLM agents and want OpenTelemetry-native tracing and evals you can self-host without losing core features.

Skip if

Skip it if you want a fully managed, zero-ops observability SaaS with white-glove enterprise support out of the box.

Phoenix, from Arize AI, is an open-source observability and evaluation platform for LLM applications and agents. It records every step an agent takes (prompts, retrievals, tool calls, outputs) via native OpenTelemetry instrumentation, then lets you score those traces with evaluation runs, human annotation, or LLM-as-judge. The included experimentation workflow turns captured traces into datasets you can replay against new prompts, models, or agent versions to measure regressions before you ship.

It sits in the eval/observability tier alongside LangSmith, Langfuse, and Helicone, but leans harder on open standards: the core is ELv2-licensed with 10k+ GitHub stars and ~3M monthly downloads, and you can self-host locally, in Docker, or on Kubernetes without losing features. Arize also offers a free Phoenix Cloud tier (two instances) and a paid Arize AX product for teams that want managed scale, SSO, and enterprise support. Best fit for AI engineers building agentic systems who want trace data in their own environment instead of a closed SaaS.

Because Phoenix is framework-agnostic, it plugs into LangChain, LlamaIndex, OpenAI, Anthropic, DSPy, CrewAI and anything else that emits OTel spans, and the newer PXI agent can help triage failing traces and propose experiments. The trade-off is the usual self-host tax: you own the storage, retention, and upgrades unless you pay for cloud.

Editor's take

Phoenix is the most credible open-source answer to LangSmith and Langfuse, and the OTel-first design ages better than proprietary SDKs. The eval and experimentation primitives are real, not bolted on, and the self-host story actually works. If you're allergic to closed observability stacks, this is the default pick.

— The AI Tool Bible editorial team

Pros

✅ Genuinely open source (ELv2) with self-host parity, not a crippled OSS shell
✅ Native OpenTelemetry means no vendor lock-in for instrumentation
✅ Covers tracing, evals, annotation, and experiments in one tool
✅ Framework-agnostic: LangChain, LlamaIndex, DSPy, CrewAI, raw SDK calls all work