Skip to main content
πŸ“– The AI Tool Bible
Inspect AI preview image
Inspect AI logo

Inspect AI

βœ“ Text source-checked Oct 31, 2026

Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.

Source-checked against the vendor's site Β· rebuilt from the vendor's own pagesProfile updated Link checked How we rate

FreeΒ· Free and open source (MIT-style license); you pay only for underlying model API usage.(not source-checked)EvaluationMulti-model (not source-checked)7.2 / 10

In short

Inspect is a free, open-source framework by the UK AI Security Institute for evaluating frontier AI models. It offers 200+ pre-built evaluations, sandboxing, and multi-model support.

Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs. Inspect supports evaluations that measure coding, agentic tasks, reasoning, knowledge, behavior, and multi-modal understanding.

Inspect provides composable building blocks including datasets, agents, tools, and scorers for writing and reusing evaluations. The framework includes a collection of over 200 pre-built evaluations ready to run on any model. Inspect offers a web-based Inspect View tool for monitoring and visualizing evaluations, along with a VS Code Extension for authoring and debugging.

A sandboxing system in Inspect supports running untrusted model code in Docker, Kubernetes, Modal, Proxmox, Vagrant, and other systems via an extension API. Inspect supports agent evaluations with flexible built-in agents, multi-agent primitives, and the ability to run external agents like Claude Code, Codex CLI, and Gemini CLI.

Inspect has built-in support for over 20 model providers and supports local inference with HuggingFace, vLLM, and SGLang. Developing and running evaluations requires access to a model, which typically involves installing a Python package and providing the appropriate API key.

From the vendor's pages

According to Inspect AI: Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs. Its pages list this strength: Includes over 200 pre-built evaluations ready to run on any model. They also state this limitation: Running evaluations requires access to a model and the appropriate API key.

β€” The AI Tool Bible editorial team

Pros

  • βœ… Includes over 200 pre-built evaluations ready to run on any model.
  • βœ… Features composable building blocks that make evaluations easy to write and reuse.
  • βœ… Provides a sandboxing system for running untrusted model code in various environments.
  • βœ… Supports over 20 model providers and local inference options.
  • βœ… Includes a web-based viewer and a VS Code extension for monitoring and authoring.

Cons

  • ⚠️ Running evaluations requires access to a model and the appropriate API key.

Facts

Platforms
web, vscode-extension, cli (not source-checked)
Pricing
free - Free and open source (MIT-style license); you pay only for underlying model API usage. (not source-checked)
API available
Yes (not source-checked)
Open source
Yes (license: MIT)
GitHub stars
3,238 (checked Nov 13, 2026) (not source-checked)
Last GitHub push
Nov 13, 2026 (not source-checked)
First commit
Nov 2023 (not source-checked)
Company
UK AI Security Institute (not source-checked)
Model used
Multi-model (not source-checked)
Hacker News
2 stories Β· top 5 points Β· last Sep 30, 2026 Β· checked Nov 13, 2026
Earliest Wayback capture
2025-03-18 (earliest archived capture of the site, not a launch date)

Where this tool sits

Use cases

Coding and reasoning evaluationAgent evaluationSandboxed code executionCapture the flag challengesTool calling and integration

Frequently asked

How much does Inspect AI cost to use?
Inspect is free and open source under an MIT-style license. You only pay for the underlying model API usage required to run your evaluations.
Which AI models are compatible with Inspect?
Inspect supports over 20 model providers and local inference via HuggingFace, vLLM, and SGLang. It also runs external agents like Claude Code, Codex CLI, and Gemini CLI.
What types of evaluations can I run?
You can measure coding, agentic tasks, reasoning, knowledge, behavior, and multi-modal understanding. The framework includes over 200 pre-built evaluations ready to run on any supported model.
Does Inspect support secure code execution?
Yes, Inspect includes a sandboxing system to run untrusted model code. It supports Docker, Kubernetes, Modal, Proxmox, Vagrant, and other systems via an extension API.
What tools help with authoring and monitoring?
Inspect provides a web-based Inspect View tool for monitoring and visualizing evaluations. It also includes a VS Code Extension for authoring and debugging your evaluation scripts.

Compare Inspect AI with similar tools

All alternatives β†’

Explore related

Reviews