Skip to main content
📖 The AI Tool Bible

All AI tools

59 tools

LL

LLMEval

Evaluation · Multi-model
7.2

Research initiative from Fudan NLP Lab building evaluation frameworks for LLMs across academic disciplines and medical AI.

FreeLLM evaluationEvaluation methodology research
PR

Promptfoo

Evaluation · Multi-model
7.2

Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.

Freemium· Free tierllm-evalsred-teaming
WA

Weco AI

Evaluation · Multi-model (LLM + AIDE tree search)
7.2

Weco AI optimizes code against measurable metrics such as speed, accuracy, and cost.

Freemiumfraud-detectionprompt-engineering
AL

AlpacaEval

Evaluation · GPT-4 Preview (Nov 2024) as annotator
7.1

Automatic LLM evaluator and leaderboard that benchmarks instruction-following with length-controlled win rates.

Free· Free and open-source; pay only for the underlying OpenAI annotator API callsllm-benchmarkinginstruction-following eval
AR

Arthur

Evaluation · Multi-model
7.1

Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust.

Freemium· Free plan at $0 per month; Premium plan at $60 per monthEnforcing guardrails in LLM applicationsBuilding and governing agentic applications
FA

Fiddler AI

Evaluation · Fiddler Centor (proprietary evaluators)
7.1

The AI Control Plane for enterprises providing visibility, context, and control through evaluation, monitoring, enforcement, governance, and cost efficiency.

Enterprise· Free plan; Developer plan at $0.002 per traceMonitoring and evaluationsHallucination and toxicity protection
MA

Maxim AI

Evaluation · Multi-model
7.1

End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Freemium· Developer: Free · Professional: $29 /seat /month · Business: $49 /seat /month · Enterprise: Customagent-evaluationllm-observability
PF

Prompt Foundry

Evaluation · OpenAI + Anthropic (multi-model)
7.1

Prompt management and side-by-side LLM evaluation for OpenAI and Anthropic models.

Freemium· Free tier (10 prompts, 500 evals/mo); Pro $15/user/mo; Enterprise customprompt-managementmodel-comparison
RF

Respan (formerly Keywords AI)

Evaluation · Multi-model (500+ via gateway)
7.1

The AI router with built-in observability and automated evals.

Freemium· Free plan at $0; Team plan at $199 per month; Enterprise plan with custom pricingModel routingAgent tracing and monitoring
SL

SEAL Leaderboard

Evaluation · Multi-model (GPT, Claude, Gemini, Llama, etc.)
7.1

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Free· Free to view; paid custom evals via Scale enterprise salesmodel-selectionbenchmark-tracking
VI

VisualWebArena

Evaluation · Model-agnostic (GPT-4V, Gemini, Claude, open VLMs)
7.1

Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.

Free· Free and open source (MIT-style research release)multimodal-agent-evalweb-browsing-benchmark
LA

LangFast

Evaluation · Multi-model
7.0

No-signup LLM playground for testing, comparing, and versioning prompts against your own API keys.

Paid· One-time lifetime ~$60-$120; 14-day money-backprompt-testingprompt-versioning
LL

llmfit

Evaluation · Multi-model
7.0

A terminal tool that right-sizes LLM models to your system's RAM, CPU, and GPU.

FreeDetermining which open-source LLMs the hardware can runEstimating SSD capacity for keeping runnable models
OL

OlympicArena

Evaluation · GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, Qwen2-72B-Instruct
7.0

Free· Free, open-source research benchmarkprocess-level evaluationleakage detection
PH

Phoenix

Evaluation · Multi-model
7.0

Open-source AI observability and evaluation platform for tracing and improving AI agents.

Freemium· Free tier · from $50/moTracing and improving AI agentsBuilding evaluations to score outputs
SU

Superwise

Evaluation · Multi-model
7.0

Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.

Freemium· Free tier · from $9/mollm-guardrailsai-governance
AG

Agenta

Evaluation · Multi-model
6.9

Open-source LLMOps platform for prompt engineering, evaluation, and observability in one workspace.

Freemium· from $29/moprompt-engineeringllm-evaluation
CO

CompassRank

Evaluation · Multi-model
6.9

Public leaderboard from the OpenCompass project ranking open and closed LLMs across 100+ benchmarks.

Free· Free leaderboard; OpenCompass toolkit is Apache 2.0 open sourcellm-benchmarkingmodel-selection
IN

InfiBench

Evaluation
6.9

Free· Free and open source (CC BY-SA 4.0)Evaluating code LLMs on real-world questionsSystematic evaluation of code LLMs
MI

MixEval

Evaluation · GPT-3.5-Turbo-0125, GPT-4o-2024-05-13, Claude 3.5 Sonnet, MixEval, MixEval-Hard
6.9

FreeModel rankingEvaluating various model sizes
AA

Arena AI

Evaluation · Multi-model
6.8

Head-to-head LLM battle arena with a public leaderboard for ranking AI models.

Free· Free to use; no public paid tier listedllm-benchmarkingmodel-comparison
AA

Artificial Analysis

Evaluation · Multi-model
6.8

Independent benchmarking platform comparing AI models and inference providers across intelligence, speed, and cost.

Freemium· from $417/momodel-benchmarkingprovider-comparison
CT

Cleanlab TLM

Evaluation · Multi-model (wraps any LLM)
6.8

Cleanlab TLM scores the trustworthiness of responses from any LLM in real-time.

FreemiumAgentsChatbots
PA

Parea AI

Evaluation · Multi-model
6.8

LLM evaluation, observability, and prompt management platform for teams shipping production AI apps.

Freemium· Free (2 seats, 3k logs/mo); Team $150/mo; Enterprise customllm-evaluationprompt-management