Skip to main content
📖 The AI Tool Bible

AI cost tracking

Editorial picks for "track llm costs".

31 tools

All Evaluation →
LA

LangSmith

Evaluation · Platform (any LLM)
8.7

LangChain's eval + observability platform.

Freemium· Developer: $0 · Plus: $39 · Enterprise: Custom pricingLLM tracingevals
HE

Helicone

Evaluation · Platform (any LLM)
8.3

AI Gateway and LLM Observability platform

Freemium· Hobby plan is Free; Pro plan is $79 per month; Team plan is $799 per monthCost tracking for LLM modelsMonitoring local and cloud LLM interactions
AG

AgentOps

Agents · Multi-model
8.2

Trace, debug, and deploy reliable AI agents.

Freemium· Basic plan: $0 per month; Pro plan: starts at $40 per monthAgent observabilityDebugging
AA

Arize AI

Evaluation · Multi-model
8.2

Arize AX is an AI engineering platform for improving AI agents and applications.

Freemium· Free tier · from $50/moChatbotsRAG systems
AA

Athina AI

Evaluation · Multi-model
8.1

Collaborative LLM evaluation and observability platform for teams shipping AI features to production.

Freemium· Starter free (10k logs/mo); Pro & Enterprise customllm-evaluationprompt-management
HO

HoneyHive

Evaluation · Multi-model
8.1

OpenTelemetry-native observability and evaluation platform for LLM agents in production.

Freemium· Free tier available; paid/enterprise tiers via salesagent-observabilityllm-evaluation
WB

W&B Weave

Evaluation · Multi-model
8.1

Production observability, tracing, and evaluation for LLM and agent systems from the Weights & Biases stack.

Freemium· Free tier available; paid and enterprise plans via W&Bllm-tracingagent-observability
HE

Headroom

Agents · Model-agnostic (Anthropic, OpenAI, Vertex, Bedrock, Azure, 100+ via LiteLLM)
7.4

Headroom is a context optimization layer for LLM applications that compresses content read by AI agents.

FreeAI agent context compressionRAG retrieval optimization
BI

Bifrost

Agents · Multi-model (OpenAI, Anthropic, Bedrock, Vertex, 1000+ via providers)
7.3

Open-source AI gateway that unifies 1000+ models behind one OpenAI-compatible endpoint with failover, budgets, and MCP routing.

Freemium· Free Forever: Free · Enterprise: Contact salesllm-gatewaymulti-provider-routing
KA

Kong AI Gateway

Agents · Multi-model
7.3

Kong AI Gateway governs LLM, MCP, and agent-to-agent traffic with a single gateway.

Paid· Plus plan from $25/month plus usage, billed monthlyLLM, MCP, and A2A traffic governancePII sanitization and data leakage prevention
LA

Langfuse

Evaluation · Model-agnostic
7.3

Open-source AI engineering platform for building, monitoring, and improving LLM applications.

Freemium· Hobby plan is Free; Core plan is $29 per month; Pro plan is $199 per month; Enterprise plan is $2499 per monthAI agent tracing and evaluationPrompt management
OP

Opik

Evaluation · Multi-model
7.3

End-to-end AI observability platform for testing, shipping, and improving agentic systems.

Freemium· Free tier · from $19/moAI agent tracing and observabilityLLM evaluation
PL

Plano

Agents · Multi-model
7.2

AI-native proxy and data plane for agents handling routing, orchestration, traces, guardrails, and model routing.

Free· Open source; commercial pricing not disclosedAgent orchestrationLLM routing
PA

Portkey AI Gateway

Agents · Multi-model (1,600+ LLMs)
7.2

Freemium· Developer plan is Free Forever; Production plan is $49/monthModel routing and failoverSemantic and simple caching
AR

Arthur

Evaluation · Multi-model
7.1

Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust.

Freemium· Free plan at $0 per month; Premium plan at $60 per monthEnforcing guardrails in LLM applicationsBuilding and governing agentic applications
MA

Manifest

Agents · Multi-model
7.1

Open-source LLM router that fans your agent traffic across providers and your existing AI subscriptions.

Freemium· Free tier · from $19/mollm-routingcost-control
PO

Portkey

Agents · Multi-model
7.1

Portkey is a production platform for AI teams offering an AI Gateway, Observability, Guardrails, Governance, and Prompt Management.

Freemium· Developer plan is Free Forever; Production plan is $49/monthAI GatewayObservability
PA

Puzzlet AI

Agents · Multi-model
7.1

Git-native prompt management and observability platform for teams shipping LLM applications.

Freemiumprompt-managementllm-observability
RF

Respan (formerly Keywords AI)

Evaluation · Multi-model (500+ via gateway)
7.1

The AI router with built-in observability and automated evals.

Freemium· Free plan at $0; Team plan at $199 per month; Enterprise plan with custom pricingModel routingAgent tracing and monitoring
AI

AICamp

Writing · Multi-model (GPT-5.2, Claude 4, Gemini)
7.0

AICamp is a comprehensive AI-powered product designed to streamline the use of various artificial intelligence tools and models.

Freemium· Starter plan is $0; Business plan is $10 per user per monthcustom-agentsprompt-library
SY

SystemPrompt

Agents · Multi-model
7.0

SystemPrompt is a self-hosted control plane that sits between AI clients, model providers, and tools to manage access and audit.

Free· Evaluation and non-production use are freeModel access managementTool call governance
TE

TeamoRouter

Agents · Multi-model
7.0

AI API gateway for Claude Code, Codex, and coding agents.

Paid· Starter plan: $10.00; Basic plan: $20.00; Standard plan: $50.00Claude Code and Codex routingSpend tracking and billing
GA

Guild AI

Agents · Multi-model (bring your own)
6.9

Guild is a neutral control plane for AI agents designed to work across all models and vendors.

EnterpriseDeploying tracing and evaluationsScoping credentials and enforcing policy
PP

Price Per Token

Coding · Multi-model
6.9

Daily-updated LLM API pricing comparison across 300+ models, with calculators, leaderboards, and a free MCP server.

Free· Free; ad-supported, no signupllm-pricing-comparisontoken-cost-calculator
AA

Artificial Analysis

Evaluation · Multi-model
6.8

Independent benchmarking platform comparing AI models and inference providers across intelligence, speed, and cost.

Freemium· from $417/momodel-benchmarkingprovider-comparison
PA

Parea AI

Evaluation · Multi-model
6.8

LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.

Freemium· Free (2 seats, 3k logs/mo); Team $150/mo; Enterprise customllm-evaluationprompt-management
CL

ClevAgent

Agents
6.6

ClevAgent is a workspace to run and guide AI agents while handling efficiency and safety in the background.

Freemium· Free plan is $0
AM

AI Meter

Evaluation

Local usage meter that turns AI coding-agent tokens into estimated electricity and water consumption.

Free· Free for individuals and companies; open source under a public GitHub repo.Tracking daily token usage across Claude Code and CursorEstimating electricity draw of an AI-assisted coding session
CL

ClickHouse

RAG

ClickHouse is an open, composable platform for data-intensive workloads including real-time analytics, transactions, observability, security, and AI engineering.

FreemiumReal-time analyticsObservability for SREs and agents
HY

Hydra

Agents · Multi-provider: routes across Claude, GPT, Gemini Flash, OpenRouter-hosted models, and local Qwen via Ollama / LM Studio

Free· Open Source plan is $0.00 with unlimited seats and requests, including every feature and commandCost-optimized task routingOn-device audit ledger for AI usage
LA

LangWatch

Evaluation · Model-agnostic; supports OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Vertex AI, and any OpenTelemetry-instrumented LLM

Agent observability and evaluation platform for AI platform teams.

Freemium· Developer: €0 free forever; Growth: €29 / core-seat / monthRed teaming and adversarial simulationPrompt version control