Skip to main content
πŸ“– The AI Tool Bible

Inspect AI vs LLM Stats

A side-by-side look at pricing, capabilities, pros, cons, and our editorial scores.

Tagline
Inspect AI
Inspect is a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs.
LLM Stats
β€”
Pricing
Inspect AI
FreeΒ· Free and open source (MIT-style license); you pay only for underlying model API usage.
LLM Stats
Free
API
Inspect AI
Yes
LLM Stats
Yes
Platforms
Inspect AI
webvscode-extensioncli
LLM Stats
api
Open source
Inspect AI
Yes Β· MIT
LLM Stats
Not listed
GitHub stars
Inspect AI
3,238
checked 2026-11-13
LLM Stats
β€”
Last GitHub push
Inspect AI
2026-11-13
LLM Stats
β€”
First commit
Inspect AI
2023-11
LLM Stats
β€”
Hacker News stories
Inspect AI
2
checked 2026-11-13
LLM Stats
5
checked 2026-11-15
Earliest Wayback capture
Inspect AI
2025-03-18
earliest archived capture of the site, not a launch date
LLM Stats
2024-08-11
earliest archived capture of the site, not a launch date
Company
Inspect AI
UK AI Security Institute
LLM Stats
β€”
Model used
Inspect AI
Multi-model
LLM Stats
Multi-model
Editorial score
Inspect AI
7.2 / 10
LLM Stats
7.9 / 10
Use cases
Inspect AI
Coding and reasoning evaluationAgent evaluationSandboxed code executionCapture the flag challengesTool calling and integration
LLM Stats
β€”
Pros
Inspect AI
  • Includes over 200 pre-built evaluations ready to run on any model.
  • Features composable building blocks that make evaluations easy to write and reuse.
  • Provides a sandboxing system for running untrusted model code in various environments.
  • Supports over 20 model providers and local inference options.
  • Includes a web-based viewer and a VS Code extension for monitoring and authoring.
LLM Stats
  • Tracks 448 canonical models across major labs and inference providers.
  • Measures output speed consistently across supported provider APIs.
  • Cross-checks pricing against billing samples through the LLM Stats proxy.
  • Provides a dedicated leaderboard for open-weights models suitable for self-hosting.
Cons
Inspect AI
  • Running evaluations requires access to a model and the appropriate API key.
LLM Stats
β€”
Website
LLM Stats
llm-stats.com

Editorial score: rule-based, 0–10, from AI-assisted profile inputs (see /methodology) β€” not a user rating; β€œβ€”β€ means unscored. β€œNot listed” means we have no record of it, not that it is absent. GitHub figures and prices carry the date they were checked or captured; prices are shown as published, unconverted.

Pick Inspect AI if
  • βœ… Includes over 200 pre-built evaluations ready to run on any model.
  • βœ… Features composable building blocks that make evaluations easy to write and reuse.
  • βœ… Provides a sandboxing system for running untrusted model code in various environments.
  • βœ… Supports over 20 model providers and local inference options.
Pick LLM Stats if
  • βœ… Tracks 448 canonical models across major labs and inference providers.
  • βœ… Measures output speed consistently across supported provider APIs.
  • βœ… Cross-checks pricing against billing samples through the LLM Stats proxy.
  • βœ… Provides a dedicated leaderboard for open-weights models suitable for self-hosting.