LLM observability
Editorial picks for "best llm observability".
48 tools
Braintrust
FeaturedBraintrust is an AI observability platform for tracing production, running evals, and catching regressions before they reach users.
LangSmith
LangChain's eval + observability platform.
Weights & Biases
The ML experiment tracker, now with LLM eval features.
Helicone
AI Gateway and LLM Observability platform
AgentOps
Trace, debug, and deploy reliable AI agents.
Arize AI
Arize AX is an AI engineering platform for improving AI agents and applications.
Giskard
Continuous AI red teaming platform that detects vulnerabilities and safeguards AI agents.
Humanloop
Prompt management + evals for collaborative AI teams.
Athina AI
Collaborative LLM evaluation and observability platform for teams shipping AI features to production.
HoneyHive
OpenTelemetry-native observability and evaluation platform for LLM agents in production.
MLflow
Open-source platform for tracking, evaluating, and deploying ML models and LLM applications.
OpenAI Evals
TruLens
Open source library for evaluating and tracing AI agents.
W&B Weave
Production observability, tracing, and evaluation for LLM and agent systems from the Weights & Biases stack.
PromptHub
A community-driven platform for discovering, managing, versioning, testing, and deploying prompts.
PromptLayer
Prompt management platform for AI teams to version prompts, run LLM evals, and monitor agents in production.
Patronus
Automated LLM evaluation for hallucinations, safety, and quality.
Kong AI Gateway
Kong AI Gateway governs LLM, MCP, and agent-to-agent traffic with a single gateway.
Langfuse
Open-source AI engineering platform for building, monitoring, and improving LLM applications.
Opik
End-to-end AI observability platform for testing, shipping, and improving agentic systems.
Seldon
Kubernetes-native MLOps framework for real-time inference, observability, and compliance at scale.
Inspect AI
Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.
Plano
AI-native proxy and data plane for agents handling routing, orchestration, traces, guardrails, and model routing.
Portkey AI Gateway
Promptfoo
Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.
Wallaroo.AI
Production AI inference platform for deploying and monitoring models across cloud, on-prem, and edge.
Arthur
Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust.
Fiddler AI
The AI Control Plane for enterprises providing visibility, context, and control through evaluation, monitoring, enforcement, governance, and cost efficiency.
Maxim AI
End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.
Portkey
Portkey is a production platform for AI teams offering an AI Gateway, Observability, Guardrails, Governance, and Prompt Management.
Puzzlet AI
Git-native prompt management and observability platform for teams shipping LLM applications.
Respan (formerly Keywords AI)
The AI router with built-in observability and automated evals.
Phoenix
Open-source AI observability and evaluation platform for tracing and improving AI agents.
Superwise
Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.
SystemPrompt
SystemPrompt is a self-hosted control plane that sits between AI clients, model providers, and tools to manage access and audit.
Agenta
Open-source LLMOps platform for prompt engineering, evaluation, and observability in one workspace.
Guild AI
Guild is a neutral control plane for AI agents designed to work across all models and vendors.
TrueFoundry
Enterprise control plane for deploying, governing, and scaling agentic AI on your own infrastructure.
Cleanlab TLM
Cleanlab TLM scores the trustworthiness of responses from any LLM in real-time.
Parea AI
LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.
AI Meter
Local usage meter that turns AI coding-agent tokens into estimated electricity and water consumption.
ClickHouse
ClickHouse is an open, composable platform for data-intensive workloads including real-time analytics, transactions, observability, security, and AI engineering.
Habibi
Self-hosted generative engine optimization for tracking brand mentions across ChatGPT, Perplexity, and Gemini
Hydra
Lagotto Meter
Measure the gap between what your site claims and what an AI agent actually finds
Lakera
An AI-native security platform designed to accelerate GenAI initiatives for enterprise teams.
LangWatch
Agent observability and evaluation platform for AI platform teams.
TokenPath
TokenPath is an API that grounds generated tokens in their source documents for output from any model.