Skip to main content
📖 The AI Tool Bible

AI prompt evaluation

Editorial picks for "prompt evaluation tool".

48 tools

All Evaluation →
BR

Braintrust

Featured
Evaluation · Platform (any LLM)
8.9

Braintrust is an AI observability platform for tracing production, running evals, and catching regressions before they reach users.

Freemium· Starter plan is $0 / month; Pro plan is $249 / monthtracing productionrunning evals
LA

LangSmith

Evaluation · Platform (any LLM)
8.7

LangChain's eval + observability platform.

Freemium· Developer: $0 · Plus: $39 · Enterprise: Custom pricingLLM tracingevals
WB

Weights & Biases

Evaluation · Platform (any LLM)
8.4

The ML experiment tracker, now with LLM eval features.

Freemium· Free: $0/mo · Pro: Starts at $60/month · Enterprise: Custom plans · Personal: $0/mo · Advanced Enterprise: Custom planML experimentsLLM eval
AA

Arize AI

Evaluation · Multi-model
8.2

Arize AX is an AI engineering platform for improving AI agents and applications.

Freemium· Free tier · from $50/moChatbotsRAG systems
GI

Giskard

Evaluation · Multi-model
8.2

Continuous AI red teaming platform that detects vulnerabilities and safeguards AI agents.

FreemiumPrompt injection testingContinuous AI agent testing
HU

Humanloop

Evaluation · Platform (any LLM)
8.2

Prompt management + evals for collaborative AI teams.

Paid· From $200/mo teamprompt managementteam collab
LI

LiveBench

Evaluation · Multi-model
8.2

Contamination-free LLM benchmark that refreshes its questions monthly to keep frontier models honest.

Free· Free and open source; self-hosted evaluation runnerllm-benchmarkingmodel-selection
OP

OpenAI Playground

Writing · Multi-model (GPT-4o, GPT-4.1, o-series, DALL-E, Whisper, TTS)
8.2

OpenAI's official browser sandbox for prompting, tuning, and testing every model on the platform before you ship API code.

Paidprompt-engineeringmodel-comparison
AA

Athina AI

Evaluation · Multi-model
8.1

Collaborative LLM evaluation and observability platform for teams shipping AI features to production.

Freemium· Starter free (10k logs/mo); Pro & Enterprise customllm-evaluationprompt-management
BF

Berkeley Function-Calling Leaderboard

Evaluation · Multi-model
8.1

Open benchmark from UC Berkeley that ranks LLMs on real-world tool-use and function-calling accuracy.

Free· Free and open source; you pay only for inference when reproducing runs.function-calling evaltool-use benchmarking
HO

HoneyHive

Evaluation · Multi-model
8.1

OpenTelemetry-native observability and evaluation platform for LLM agents in production.

Freemium· Free tier available; paid/enterprise tiers via salesagent-observabilityllm-evaluation
ML

MLflow

Evaluation · Multi-model
8.1

Open-source platform for tracking, evaluating, and deploying ML models and LLM applications.

Free· Free and open source (Apache 2.0); managed offering via Databricksllm-evaluationexperiment-tracking
OE

OpenAI Evals

Evaluation · OpenAI GPT models (extensible)
8.1

FreeEvaluating LLMs and LLM systemsTesting different dimensions of OpenAI models
TR

TruLens

Evaluation · Multi-model (LLM-as-judge)
8.1

Open source library for evaluating and tracing AI agents.

FreeAI agent evaluation and tracingAgent tool selection and plan adherence
WB

W&B Weave

Evaluation · Multi-model
8.1

Production observability, tracing, and evaluation for LLM and agent systems from the Weights & Biases stack.

Freemium· Free tier available; paid and enterprise plans via W&Bllm-tracingagent-observability
PR

PromptHub

Writing · Multi-model (OpenAI, Anthropic, Google, Meta, Mistral, Bedrock, Azure)
8.0

A community-driven platform for discovering, managing, versioning, testing, and deploying prompts.

Freemium· Free plan; Pro plan is $12/month; Team plan is $20/user/monthPrompt management and versioningPrompt evaluation
RI

Rivet

Agents · Multi-model
8.0

Visual programming environment for building AI agents with LLMs.

FreeBuilding AI agentsPrompt chaining
LS

LLM Stats

Evaluation · Multi-model
7.9

Free
PR

PromptLayer

Evaluation · Platform (any LLM)
7.9

Prompt management platform for AI teams to version prompts, run LLM evals, and monitor agents in production.

Freemium· Free: $0 per month; Pro: $49 per month; Team: $500 per monthPrompt managementAgent observability
PA

Patronus

Evaluation · Platform (any LLM)
7.8

Automated LLM evaluation for hallucinations, safety, and quality.

Paid· Individual: Free · Base: $25 · Enterprise: Contact us for Pricinghallucination detectionsafety
LA

Langfuse

Evaluation · Model-agnostic
7.3

Open-source AI engineering platform for building, monitoring, and improving LLM applications.

Freemium· Hobby plan is Free; Core plan is $29 per month; Pro plan is $199 per month; Enterprise plan is $2499 per monthAI agent tracing and evaluationPrompt management
OP

Opik

Evaluation · Multi-model
7.3

End-to-end AI observability platform for testing, shipping, and improving agentic systems.

Freemium· Free tier · from $19/moAI agent tracing and observabilityLLM evaluation
IA

Inspect AI

Evaluation · Multi-model
7.2

Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.

Free· Free and open source (MIT-style license); you pay only for underlying model API usage.Coding and reasoning evaluationAgent evaluation
KA

Kiln AI

Evaluation · Multi-model
7.2

Open-source workbench for building, evaluating, and fine-tuning AI agents across 190+ models.

Freemium· Free Individual tier; Team (request access); Enterprise (custom)llm-evaluationfine-tuning
PR

Promptfoo

Evaluation · Multi-model
7.2

Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.

Freemium· Free tierllm-evalsred-teaming
WA

Weco AI

Evaluation · Multi-model (LLM + AIDE tree search)
7.2

Weco AI optimizes code against measurable metrics such as speed, accuracy, and cost.

Freemiumfraud-detectionprompt-engineering
AL

AlpacaEval

Evaluation · GPT-4 Preview (Nov 2024) as annotator
7.1

Automatic LLM evaluator and leaderboard that benchmarks instruction-following with length-controlled win rates.

Free· Free and open-source; pay only for the underlying OpenAI annotator API callsllm-benchmarkinginstruction-following eval
MA

Maxim AI

Evaluation · Multi-model
7.1

End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Freemium· Developer: Free · Professional: $29 /seat /month · Business: $49 /seat /month · Enterprise: Customagent-evaluationllm-observability
PF

Prompt Foundry

Evaluation · OpenAI + Anthropic (multi-model)
7.1

Prompt management and side-by-side LLM evaluation for OpenAI and Anthropic models.

Freemium· Free tier (10 prompts, 500 evals/mo); Pro $15/user/mo; Enterprise customprompt-managementmodel-comparison
PA

Puzzlet AI

Agents · Multi-model
7.1

Git-native prompt management and observability platform for teams shipping LLM applications.

Freemiumprompt-managementllm-observability
RF

Respan (formerly Keywords AI)

Evaluation · Multi-model (500+ via gateway)
7.1

The AI router with built-in observability and automated evals.

Freemium· Free plan at $0; Team plan at $199 per month; Enterprise plan with custom pricingModel routingAgent tracing and monitoring
SL

SEAL Leaderboard

Evaluation · Multi-model (GPT, Claude, Gemini, Llama, etc.)
7.1

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Free· Free to view; paid custom evals via Scale enterprise salesmodel-selectionbenchmark-tracking
VI

VisualWebArena

Evaluation · Model-agnostic (GPT-4V, Gemini, Claude, open VLMs)
7.1

Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.

Free· Free and open source (MIT-style research release)multimodal-agent-evalweb-browsing-benchmark
LA

LangFast

Evaluation · Multi-model
7.0

No-signup LLM playground for testing, comparing, and versioning prompts against your own API keys.

Paid· One-time lifetime ~$60-$120; 14-day money-backprompt-testingprompt-versioning
PH

Phoenix

Evaluation · Multi-model
7.0

Open-source AI observability and evaluation platform for tracing and improving AI agents.

Freemium· Free tier · from $50/moTracing and improving AI agentsBuilding evaluations to score outputs
PR

Prompteams

Writing
7.0

Prompteams manages AI prompts, provides a playground and LLM testing, and offers version control and auto-generated APIs for team integration.

Freemium· from 100/moprompt versioningprompt testing
AG

Agenta

Evaluation · Multi-model
6.9

Open-source LLMOps platform for prompt engineering, evaluation, and observability in one workspace.

Freemium· from $29/moprompt-engineeringllm-evaluation
CO

CompassRank

Evaluation · Multi-model
6.9

Public leaderboard from the OpenCompass project ranking open and closed LLMs across 100+ benchmarks.

Free· Free leaderboard; OpenCompass toolkit is Apache 2.0 open sourcellm-benchmarkingmodel-selection
IZ

Izlo

Agents · Model-agnostic
6.9

Izlo provides prompt control for teams to unlock prompts trapped in code, documents, and spreadsheets.

Paid· Solo: $20 per month; Pro: $25 per user/month; Enterprise: $39 per user/monthprompt managementversion control
MI

MixEval

Evaluation · GPT-3.5-Turbo-0125, GPT-4o-2024-05-13, Claude 3.5 Sonnet, MixEval, MixEval-Hard
6.9

FreeModel rankingEvaluating various model sizes
AA

Arena AI

Evaluation · Multi-model
6.8

Head-to-head LLM battle arena with a public leaderboard for ranking AI models.

Free· Free to use; no public paid tier listedllm-benchmarkingmodel-comparison
PA

Parea AI

Evaluation · Multi-model
6.8

LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.

Freemium· Free (2 seats, 3k logs/mo); Team $150/mo; Enterprise customllm-evaluationprompt-management
PR

PromptPerfect

Writing · Multi-model (GPT, Claude, Stable Diffusion, Midjourney targets)
6.8

Prompt optimizer from Jina AI that rewrites and stress-tests prompts across major LLMs.

Freemium· Free: Free · Pro: $19.99 · Pro Max: $99.99 · Enterprise: Contact salesprompt-optimizationprompt-engineering
MP

Magic Potion

Writing · Multi-model
6.5

Visual drag-and-drop prompt editor for crafting, organizing, and reusing prompts across OpenAI, Anthropic, and Google models.

Freemiumprompt-engineeringprompt-library
AW

AI World Bakeoff

Evaluation · Ten frontier coding models (specific list not published on landing page; includes at least one Claude Opus generation referenced as 'Opus 5')

Ten AI coding models build three identical explorable 3D worlds in a single autonomous shot.

Free
LM

Lagotto Meter

Evaluation

Measure the gap between what your site claims and what an AI agent actually finds

Free· Free (currently in public beta)AI-visibility audit of a landing pageFact-vs-claim gap analysis for marketing copy
LA

LangWatch

Evaluation · Model-agnostic; supports OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Vertex AI, and any OpenTelemetry-instrumented LLM

Agent observability and evaluation platform for AI platform teams.

Freemium· Developer: €0 free forever; Growth: €29 / core-seat / monthRed teaming and adversarial simulationPrompt version control
MO

ModelBias

Evaluation · 100 models across Anthropic, OpenAI, Google, DeepSeek, Meta, xAI, Mistral, Qwen and others (via OpenRouter)

100 AI models, 100 simple questions, asked 3 times each, with 30,000 answers to explore.

FreeComparing hidden defaults, associations, and biases of AI modelsUsing the dataset in one's own project or research