AgentDojo

Spec sheet

Purpose

AgentDojo is a dynamic environment for evaluating prompt injection attacks and defenses for LLM agents.Read 8 Oct 2026 · agentdojo.spylab.ai

License

The project's GitHub page identifies its license as MIT.Read 8 Oct 2026 · github.com

Affiliations

The listed authors are affiliated with ETH Zurich and Invariant Labs.Read 8 Oct 2026 · agentdojo.spylab.ai

API status

The package API is still under development and may change.Read 8 Oct 2026 · agentdojo.spylab.ai

Benchmark

The benchmark script runs selected models, defenses, attacks, task suites, and tasks.Read 8 Oct 2026 · agentdojo.spylab.ai

Contribute results

Users can submit benchmark results by opening a repository pull request that includes a brief description and an implementation.Read 8 Oct 2026 · agentdojo.spylab.ai

Custom attacks

Users can create and plug in custom attacks that target a pipeline, task suite, user task, and injection task.Read 8 Oct 2026 · agentdojo.spylab.ai

Custom benchmarks

Users can define task suites and tasks to extend the provided benchmarks or create a new benchmark from scratch.Read 8 Oct 2026 · agentdojo.spylab.ai

Custom pipelines

Pipeline components can be combined into custom agent pipelines, including defense components.Read 8 Oct 2026 · agentdojo.spylab.ai

Defense research

The results page reports model, defense, and attack combinations and says it is not a leaderboard because not all combinations were tested.Read 8 Oct 2026 · agentdojo.spylab.ai

Function runtime

Its functions runtime defines and runs Python functions for use with function-calling LLMs, including functions that operate on stateful environments.Read 8 Oct 2026 · agentdojo.spylab.ai

Install

The quickstart installs AgentDojo with `pip install agentdojo`.Read 8 Oct 2026 · agentdojo.spylab.ai

Makers

The project page lists authors affiliated with ETH Zurich and Invariant Labs.Read 8 Oct 2026 · agentdojo.spylab.ai

Model integrations

Documented tool calling LLM implementations include OpenAI GPT, Anthropic Claude, Google Gemini, Cohere Command, and a prompting interface for other models.Read 8 Oct 2026 · agentdojo.spylab.ai

Model providers

Documented tool-calling LLM implementations support OpenAI, Anthropic, Google, and Cohere; the docs also describe using Llama 3 70B through TogetherAI’s API.Read 8 Oct 2026 · agentdojo.spylab.ai

Pipeline components

Agent pipelines combine elements such as system messages, LLMs, tool execution, and defense components.Read 8 Oct 2026 · agentdojo.spylab.ai

Prompt injection detector

The prompt injection detector requires installing the `transformers` extra.Read 8 Oct 2026 · agentdojo.spylab.ai

Research use

The project asks research users to consider citing its 2024 paper.Read 8 Oct 2026 · agentdojo.spylab.ai

Results limitation

The results page says its results are not a leaderboard because not all models were tested with all attacks and defenses.Read 8 Oct 2026 · agentdojo.spylab.ai

Tool runtime

Its functions runtime registers and runs Python functions for use with function calling LLMs, including functions that use stateful environments.Read 8 Oct 2026 · agentdojo.spylab.ai

Trace inspection

The results page says users can inspect run traces, including agent trajectories, attacks, and defenses, using Invariant Explorer.Read 8 Oct 2026 · agentdojo.spylab.ai

Use case

The documentation presents AgentDojo as a framework for researchers to evaluate attacks and defenses and build benchmarks.Read 8 Oct 2026 · agentdojo.spylab.ai

Where Specifiction places it

• AI Security Testing Tools#12 of 297.8