LLM monitoring
Editorial picks for "best llm monitoring".
39 tools

ScrapeGraphAI
LLM-driven web scraping API that turns natural-language prompts into structured JSON.

Dataiku
Enterprise AI platform unifying data, ML, LLMs, and agents under one governed workflow.

AgentOps
Observability and debugging platform purpose-built for AI agents, with time-travel replay and cost tracking across 400+ LLMs.

Arize AI
Enterprise observability and evaluation platform for LLM agents and generative AI applications.

Browser Use
Open-source browser automation harness and cloud platform for LLM agents that drive real websites.

Feast
Open-source feature store that serves consistent features to ML training and online inference, with RAG vector search built in.

Great Expectations
Open-source data quality framework for validating the datasets that feed your ML and analytics pipelines.

Sim
Open-source visual workspace for building, deploying, and monitoring AI agents.

Athina AI
Collaborative LLM evaluation and observability platform for teams shipping AI features to production.

OpenMetadata
Open-source metadata platform that gives AI agents a semantic context graph over your data stack.

OpenBB
Open-source financial workspace where analysts and AI agents share the same governed data.

Opik
Open-source LLM observability and evaluation platform for debugging and monitoring AI agents in production.

Seldon
Kubernetes-native MLOps platform for deploying and orchestrating ML and generative AI models in production.

Wallaroo.AI
Production AI inference platform for deploying and monitoring models across cloud, on-prem, and edge.

Arthur
Open-source toolkit for testing, tracing, and monitoring production AI agents.

Fiddler AI
Enterprise AI observability and guardrails platform for monitoring agents, LLMs, and ML models in production.

Iguazio
Enterprise MLOps and GenAI platform for taking models from notebook to production at scale.

KNIME
Visual node-based data science platform with built-in connectors for OpenAI, Anthropic, Gemini, and local LLMs.

Maxim AI
End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Respan (formerly Keywords AI)
LLM engineering platform combining a multi-model gateway with tracing, evals, and prompt management.

Tableau
Salesforce-owned BI platform that bolted generative AI onto enterprise dashboards via Tableau Pulse and Tableau Agent.

Bisheng
Open-source enterprise AgentOps platform for building, orchestrating, and governing LLM agents at scale.

Pagerly
Slack-native on-call scheduling and incident response with AI-assisted rotations.

ReBillion.ai
AI transaction coordinator that extracts dates, deadlines, and compliance rules from real estate contracts.

AI Meter
Local usage meter that turns AI coding-agent tokens into estimated electricity and water consumption.

AutotuneLLM
An open-source optimization layer that sits between your app and Ollama to squeeze more performance out of local LLMs.

Browserbase
Headless browser infrastructure for AI agents

Caveman
The token-efficient stack for agent-native development

Competiflow
Structured competitor monitoring with AI severity scoring, delivered to your dashboard, Slack or agents via MCP.

Decagon
Enterprise AI concierge platform for voice, chat, and email support agents.
Firecrawl MCP Server
MCP server that gives Claude, Cursor, and other AI agents native web scrape, crawl, search, and extract tools via Firecrawl.
Home Assistant MCP
Open-source MCP server that lets any LLM control and observe a Home Assistant smart home in real time.

Lakera
Runtime security and guardrails for GenAI apps, agents, and RAG systems.

Neverbell
Market access skill that lets AI agents monitor and trade real markets
Postgres MCP Pro
Open-source Postgres MCP server with deterministic health checks, index tuning, and safe SQL execution.
Reddit MCP (adhikasp/mcp-reddit)
MCP server that lets Claude and other LLM clients fetch and analyse Reddit threads

Stagehand
The SDK for browser agents

Tavily
One secure API for real-time web access for AI agents

Weather MCP Server
MCP server that gives Claude live weather, climate, terrain, and hazard data for any location on Earth.