LLM observability
Editorial picks for "best llm observability".
48 tools
Elasticsearch Vector Search
Hybrid vector + keyword search in the enterprise-grade Elasticsearch engine
LangSmith
LangChain's eval + observability platform.
Quivr
Open-source RAG framework for building custom AI assistants over your own documents in a few lines of Python.
Semantic Kernel
Microsoft's open-source SDK for wiring LLMs, plugins, and agents into enterprise .NET, Python, and Java apps.
Helicone
Open-source LLM observability — one-line proxy install.
Vanna.ai
Open-source text-to-SQL agent that learns your schema and writes queries against your real warehouse.
AgentOps
Observability and debugging platform purpose-built for AI agents, with time-travel replay and cost tracking across 400+ LLMs.
Arize AI
Enterprise observability and evaluation platform for LLM agents and generative AI applications.
BentoML
Open-source framework and managed platform for serving and scaling AI models in production.
Athina AI
Collaborative LLM evaluation and observability platform for teams shipping AI features to production.
HoneyHive
OpenTelemetry-native observability and evaluation platform for LLM agents in production.
Mastra
Open-source TypeScript framework for building, deploying, and observing AI agents and workflows.
MLflow
Open-source platform for tracking, evaluating, and deploying ML models and LLM applications.
TruLens
Open-source evaluation and tracing framework for LLM apps and agents, built on OpenTelemetry.
.png){kind=small}
W&B Weave
Production observability, tracing, and evaluation for LLM and agent systems from the Weights & Biases stack.
CAMEL-AI
Open-source Python framework for building multi-agent systems and synthetic data pipelines.
PromptLayer
Lightweight prompt logging + management for OpenAI/Claude apps.
Headroom
Open-source context compression layer that strips 70-95% of boilerplate before it hits your LLM.
AgentMemory
Open-source persistent memory runtime for AI coding agents, with hybrid retrieval and zero external dependencies.
Bifrost
Open-source AI gateway that unifies 1000+ models behind one OpenAI-compatible endpoint with failover, budgets, and MCP routing.
Kong AI Gateway
Enterprise API gateway extended to route, govern, and observe LLM and agent traffic across providers.
Langfuse
Open-source LLM observability, prompt management, and evaluation in one platform.
Opik
Open-source LLM observability and evaluation platform for debugging and monitoring AI agents in production.
Chainlit
Open-source Python framework for building production-grade conversational AI interfaces in minutes.
MCP Toolbox for Databases
Open-source MCP server that wires AI agents and IDEs straight into 50+ production databases with auth, pooling, and observability baked in.
Plano
Envoy-based data plane for AI agents that handles routing, guardrails, and observability outside your app code.
Portkey AI Gateway
Open-source AI gateway that routes a single API call across 1,600+ LLMs with caching, fallbacks, and observability.
WeKnora
Tencent's open-source RAG framework that turns raw documents into a queryable knowledge base, ReAct agent, and self-maintaining wiki.
Arthur
Open-source toolkit for testing, tracing, and monitoring production AI agents.
ElizaOS
Open-source agentic OS for building and orchestrating multi-agent swarms across Discord, Telegram, X, and onchain.
Fiddler AI
Enterprise AI observability and guardrails platform for monitoring agents, LLMs, and ML models in production.
iii
Self-hosted distributed runtime for multi-language workers and AI agents under one protocol.
Maxim AI
End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.
Portkey
Production LLM gateway with observability, guardrails, and prompt management for teams shipping AI in anger.
Puzzlet AI
Git-native prompt management and observability platform for teams shipping LLM applications.
Respan (formerly Keywords AI)
LLM engineering platform combining a multi-model gateway with tracing, evals, and prompt management.
Kotaemon
Open-source RAG UI for chatting with your own documents, locally or self-hosted.
LangFast
No-signup LLM playground for testing, comparing, and versioning prompts against your own API keys.
Phoenix
Open-source LLM and agent observability platform with tracing, evals, and experimentation built on OpenTelemetry.
Superwise
Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.
Agenta
Open-source LLMOps platform for prompt engineering, evaluation, and observability in one workspace.
Izlo
Prompt management platform with version control, collaboration, and an API for production deployment.
TrueFoundry
Enterprise control plane for deploying, governing, and scaling agentic AI on your own infrastructure.
Parea AI
LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.
Browserbase
Headless browser infrastructure for AI agents
Caveman
The token-efficient stack for agent-native development
ClickHouse
The open-source columnar database powering real-time analytics — and, increasingly, LLM observability and RAG backends.
Grafana MCP
Official Grafana Labs MCP server — dashboards, Prometheus, Loki, alerts and incidents in your LLM client