Skip to main content
📖 The AI Tool Bible

Best AI tools for safety testing

19 tools in the Evaluation category, filtered to safety testing.

All Evaluation →
AA

Arize AI

Evaluation · Multi-model
8.2

Arize AX is an AI engineering platform for improving AI agents and applications.

Freemium· Free tier · from $50/moChatbotsRAG systems
GI

Giskard

Evaluation · Multi-model
8.2

Continuous AI red teaming platform that detects vulnerabilities and safeguards AI agents.

FreemiumPrompt injection testingContinuous AI agent testing
OE

OpenAI Evals

Evaluation · OpenAI GPT models (extensible)
8.1

FreeEvaluating LLMs and LLM systemsTesting different dimensions of OpenAI models
TR

TruLens

Evaluation · Multi-model (LLM-as-judge)
8.1

Open source library for evaluating and tracing AI agents.

FreeAI agent evaluation and tracingAgent tool selection and plan adherence
PA

Patronus

Evaluation · Platform (any LLM)
7.8

Automated LLM evaluation for hallucinations, safety, and quality.

Paid· Individual: Free · Base: $25 · Enterprise: Contact us for Pricinghallucination detectionsafety
IA

Inspect AI

Evaluation · Multi-model
7.2

Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.

Free· Free and open source (MIT-style license); you pay only for underlying model API usage.Coding and reasoning evaluationAgent evaluation
LL

LLMEval

Evaluation · Multi-model
7.2

Research initiative from Fudan NLP Lab building evaluation frameworks for LLMs across academic disciplines and medical AI.

FreeLLM evaluationEvaluation methodology research
PR

Promptfoo

Evaluation · Multi-model
7.2

Open-source eval and red-teaming framework for LLM apps, prompts, and RAG pipelines.

Freemium· Free tierllm-evalsred-teaming
AR

Arthur

Evaluation · Multi-model
7.1

Arthur unites security, finance, and the teams shipping AI on one control plane for AI you can trust.

Freemium· Free plan at $0 per month; Premium plan at $60 per monthEnforcing guardrails in LLM applicationsBuilding and governing agentic applications
FA

Fiddler AI

Evaluation · Fiddler Centor (proprietary evaluators)
7.1

The AI Control Plane for enterprises providing visibility, context, and control through evaluation, monitoring, enforcement, governance, and cost efficiency.

Enterprise· Free plan; Developer plan at $0.002 per traceMonitoring and evaluationsHallucination and toxicity protection
MA

Maxim AI

Evaluation · Multi-model
7.1

End-to-end evaluation, simulation, and observability platform for shipping production-grade AI agents.

Freemium· Developer: Free · Professional: $29 /seat /month · Business: $49 /seat /month · Enterprise: Customagent-evaluationllm-observability
SL

SEAL Leaderboard

Evaluation · Multi-model (GPT, Claude, Gemini, Llama, etc.)
7.1

Private, expert-graded leaderboards from Scale AI that rank frontier LLMs on domains contaminated public benchmarks can no longer measure.

Free· Free to view; paid custom evals via Scale enterprise salesmodel-selectionbenchmark-tracking
VI

VisualWebArena

Evaluation · Model-agnostic (GPT-4V, Gemini, Claude, open VLMs)
7.1

Open benchmark for evaluating multimodal web agents on realistic visual browsing tasks.

Free· Free and open source (MIT-style research release)multimodal-agent-evalweb-browsing-benchmark
OL

OlympicArena

Evaluation · GPT-4o, Claude-3.5-Sonnet, Doubao-Pro-32k, DeepSeek-Coder-V2, Qwen2-72B-Instruct
7.0

Free· Free, open-source research benchmarkprocess-level evaluationleakage detection
SU

Superwise

Evaluation · Multi-model
7.0

Agentic management platform for runtime guardrails, policy enforcement, and observability across LLM agents.

Freemium· Free tier · from $9/mollm-guardrailsai-governance
CT

Cleanlab TLM

Evaluation · Multi-model (wraps any LLM)
6.8

Cleanlab TLM scores the trustworthiness of responses from any LLM in real-time.

FreemiumAgentsChatbots
LA

Lakera

Evaluation · Proprietary in-house classifiers; model-agnostic (works in front of GPT-4o, Claude, Gemini, Llama, and custom LLMs)

An AI-native security platform designed to accelerate GenAI initiatives for enterprise teams.

FreemiumPreventing prompt injections, data leakage, and jailbreaksShadow AI discovery across apps and browsers
MO

ModelBias

Evaluation · 100 models across Anthropic, OpenAI, Google, DeepSeek, Meta, xAI, Mistral, Qwen and others (via OpenRouter)

100 AI models, 100 simple questions, asked 3 times each, with 30,000 answers to explore.

FreeComparing hidden defaults, associations, and biases of AI modelsUsing the dataset in one's own project or research
MO

ModelFuzz

Evaluation · Model-agnostic; works with any OpenAI-compatible endpoint (Qwen 2.5 used in official examples).

ModelFuzz secures LLM agents against prompt injection by scanning for vulnerabilities and blocking unsafe tool calls at the execution layer.

Freemium· Free / open-source (MIT) via pip. Hosted dashboard with centralized policies, audit logs and continuous scanning coming soon via waitlist (pricing not yet public).Red-teaming OpenAI-compatible endpointsBlocking unsafe tool calls