ModelFuzz

Open-source red-teaming and execution-layer defense for AI agents against prompt injection.

Freemium· Free / open-source (MIT) via pip. Hosted dashboard with centralized policies, audit logs and continuous scanning coming soon via waitlist (pricing not yet public).EvaluationModel-agnostic; works with any OpenAI-compatible endpoint (Qwen 2.5 used in official examples).

Visit website →

Best for

Python engineering teams shipping agentic LLM products who need real, execution-layer guardrails on tool calls plus a repeatable adversarial scan they can run in CI against any OpenAI-compatible endpoint.

Skip if

Non-Python stacks, teams that only need chat-style content moderation with no tool use, or buyers who want a fully managed enterprise security platform with SLAs today rather than a waitlist.

ModelFuzz is an open-source security toolkit that hardens LLM-powered agents against prompt injection and unsafe tool use. Instead of relying on prompt-level filtering, which attackers can talk around, it works at the execution layer: it inspects the arguments an agent is about to hand to a function or external API and blocks the call when it violates a configured policy.

The project ships two complementary halves. The offense side is a red-teaming scanner that hits any OpenAI-compatible endpoint with a library of adversarial prompts and reports which ones successfully jailbreak the model, exfiltrate data, or coerce the agent into unintended tool calls. The defense side is a lightweight Python decorator you wrap around your tool functions; before each invocation it validates arguments against pluggable policy engines such as URLAllowList, blocks anything unsafe, and writes a detailed audit log.

Typical workflows include: pointing the scanner at a staging agent to discover indirect-injection paths coming through poisoned emails, retrieved documents, or web pages; wrapping browsing, shell, filesystem, or HTTP-request tools with policies that constrain domains, file paths, or payload shapes; running the scanner in CI so new tools or model swaps don't silently regress agent safety; and pairing it with observability stacks to alert on blocked calls.

It is aimed at engineering teams shipping agentic products (RAG chatbots with tool use, browsing agents, coding copilots, internal automation) who need something more concrete than model system-prompt hardening. Being MIT-licensed and pip-installable, it drops into existing Python agent stacks without pulling in a vendor. A hosted dashboard for centralized policy management, longitudinal audit logs, and continuous scans is on the roadmap behind a waitlist.

Editor's take

I like that ModelFuzz treats agent safety as an execution problem, not a prompt-engineering problem. The decorator model is the right shape: cheap to adopt, hard for a jailbroken model to bypass, and honest about what it blocked. It is still early, and the built-in policy set is thin, but as a free, self-hostable red-team-plus-runtime layer it is one of the more pragmatic offerings in the agent-security space.

— The AI Tool Bible editorial team

Pros

✅ Execution-layer enforcement blocks unsafe tool calls even when the model is jailbroken.
✅ Combines red-team scanning and runtime defense in a single project, so findings map directly to policies.
✅ MIT-licensed and pip-installable; no lock-in and no data leaves your environment.
✅ OpenAI-compatible scanner works against any endpoint you can point at, including local models like Qwen 2.5.
✅ Simple Python decorator API integrates with existing agent code without a framework rewrite.
✅ Detailed audit logs of blocked calls make incident triage and CI regression tests straightforward.

Cons

⚠️ Python-only defense integration; teams on Node, Go, or Ruby agent stacks are left out for now.
⚠️ Policy library is still small (URLAllowList and a handful of others); complex apps will need to write custom validators.
⚠️ Hosted dashboard, centralized policies, and continuous scanning are waitlist-only, so self-hosting is required today.
⚠️ Scanner coverage depends on its adversarial prompt set; novel or domain-specific attacks may slip through without custom seeds.
⚠️ Early-stage project with limited third-party benchmarks and a small community compared to established red-team suites.

Use cases

Red-teaming OpenAI-compatible agent endpointsBlocking indirect prompt injection via retrieved documentsURL allow-listing for browsing agentsPolicy-guarded tool calls for RAG chatbotsCI regression tests for agent safetyAudit logging of blocked agent actionsHardening internal automation agentsPre-deployment vulnerability scanning of LLM apps