Skip to main content
📖 The AI Tool Bible

AI safety testing

Editorial picks for "ai safety eval".

22 tools

All Evaluation →
AA

Arize AI

Evaluation · Multi-model
8.2

Arize AX is an AI engineering platform for improving AI agents and applications.

Freemium· Free tier · from $50/moChatbotsRAG systems
GI

Giskard

Evaluation · Multi-model
8.2

Continuous AI red teaming platform that detects vulnerabilities and safeguards AI agents.

FreemiumPrompt injection testingContinuous AI agent testing
OE

OpenAI Evals

Evaluation · OpenAI GPT models (extensible)
8.1

FreeEvaluating LLMs and LLM systemsTesting different dimensions of OpenAI models
TR

TruLens

Evaluation · Multi-model (LLM-as-judge)
8.1

Open source library for evaluating and tracing AI agents.

FreeAI agent evaluation and tracingAgent tool selection and plan adherence
PA

Patronus

Evaluation · Platform (any LLM)
7.8

Automated LLM evaluation for hallucinations, safety, and quality.

Paid· Individual: Free · Base: $25 · Enterprise: Contact us for Pricinghallucination detectionsafety
IA

Inspect AI

Evaluation · Multi-model
7.2

Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.

Free· Free and open source (MIT-style license); you pay only for underlying model API usage.Coding and reasoning evaluationAgent evaluation
PR

Promptfoo

Evaluation · Multi-model
7.2

Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.

Freemium· Free tierllm-evalsred-teaming
AR

Arthur

Evaluation · Multi-model
7.1

Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust.

Freemium· Free plan at $0 per month; Premium plan at $60 per monthEnforcing guardrails in LLM applicationsBuilding and governing agentic applications
FA

Fiddler AI

Evaluation · Fiddler Centor (proprietary evaluators)
7.1

The AI Control Plane for enterprises providing visibility, context, and control through evaluation, monitoring, enforcement, governance, and cost efficiency.

Enterprise· Free plan; Developer plan at $0.002 per traceMonitoring and evaluationsHallucination and toxicity protection
MA

Maxim AI

Evaluation · Multi-model
7.1

End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Freemium· Developer: Free · Professional: $29 /seat /month · Business: $49 /seat /month · Enterprise: Customagent-evaluationllm-observability
SL

SEAL Leaderboard

Evaluation · Multi-model (GPT, Claude, Gemini, Llama, etc.)
7.1

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Free· Free to view; paid custom evals via Scale enterprise salesmodel-selectionbenchmark-tracking
VI

VisualWebArena

Evaluation · Model-agnostic (GPT-4V, Gemini, Claude, open VLMs)
7.1

Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.

Free· Free and open source (MIT-style research release)multimodal-agent-evalweb-browsing-benchmark
SU

Superwise

Evaluation · Multi-model
7.0

Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.

Freemium· Free tier · from $9/mollm-guardrailsai-governance
GA

Guild AI

Agents · Multi-model (bring your own)
6.9

Guild is a neutral control plane for AI agents designed to work across all models and vendors.

EnterpriseDeploying tracing and evaluationsScoping credentials and enforcing policy
CT

Cleanlab TLM

Evaluation · Multi-model (wraps any LLM)
6.8

Cleanlab TLM scores the trustworthiness of responses from any LLM in real-time.

FreemiumAgentsChatbots
PG

Prediction Guard

Agents · Multi-model
6.6

Self-hosted AI control plane that lets regulated enterprises govern models, agents, and MCP servers behind their firewall.

Enterprise· Contact salesai-governanceself-hosted-llm
AX

Axtary

Agents

Content authorization for AI agents that checks the exact diff, message, query, or tool payload before a connector executes.

Freemium· Local plan: $0; Founding Team plan: $499 per month after a 30-day pilotGitHub pull request creation with specific constraintsSlack message posting with channel scope and recipient step-up
CW

Cloud World Model

Agents

Simulate cloud infrastructure cost, latency, errors, and resilience across AWS, GCP, Azure, OCI, and DigitalOcean before deployment.

Freemium· Small credit pack: 10,000 credits for $9; Medium credit pack: 100,000 credits for $49; Large credit pack: 1,000,000 credits for $299Multi-cloud cost analysisSimulated failure experiments
LA

Lakera

Evaluation · Proprietary in-house classifiers; model-agnostic (works in front of GPT-4o, Claude, Gemini, Llama, and custom LLMs)

An AI-native security platform designed to accelerate GenAI initiatives for enterprise teams.

FreemiumPreventing prompt injections, data leakage, and jailbreaksShadow AI discovery across apps and browsers
LA

LangWatch

Evaluation · Model-agnostic; supports OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, Vertex AI, and any OpenTelemetry-instrumented LLM

Agent observability and evaluation platform for AI platform teams.

Freemium· Developer: €0 free forever; Growth: €29 / core-seat / monthRed teaming and adversarial simulationPrompt version control
MO

ModelBias

Evaluation · 100 models across Anthropic, OpenAI, Google, DeepSeek, Meta, xAI, Mistral, Qwen and others (via OpenRouter)

100 AI models, 100 simple questions, asked 3 times each, with 30,000 answers to explore.

FreeComparing hidden defaults, associations, and biases of AI modelsUsing the dataset in one's own project or research
MO

ModelFuzz

Evaluation · Model-agnostic; works with any OpenAI-compatible endpoint (Qwen 2.5 used in official examples).

ModelFuzz secures LLM agents against prompt injection by scanning for vulnerabilities and blocking unsafe tool calls at the execution layer.

Freemium· Free / open-source (MIT) via pip. Hosted dashboard with centralized policies, audit logs and continuous scanning coming soon via waitlist (pricing not yet public).Red-teaming OpenAI-compatible endpointsBlocking unsafe tool calls