Skip to content
SPECIFICTION
Get listed

PyRIT

microsoft.github.io

Free plan WebLinuxAPISelf-hosted
Visit microsoft.github.io
Spec score 7.5 #18 of 29 in LLM Evaluation Tools

What it is

PyRIT is a free, MIT-licensed framework for security professionals and engineers assessing risks in generative AI systems. It supports single-turn and multi-turn red-team strategies such as Crescendo, TAP, and Skeleton Key. Scenario evaluations can be repeated across hundreds of objectives, including content harms, psychosocial risks, and data leakage. CoPyRIT provides a web interface for interacting with AI systems, recording findings, and collaborating with a team. Documented targets include OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. PyRIT stores conversations, scores, and attack results in SQLite or Azure SQL and can export results. Scoring supports true/false, Likert, classification, and custom scorers, including LLM-based and Azure AI Content Safety methods. Scanner, GUI, Framework, pyrit_scan, and pyrit_shell modes are documented. Installation is available through Docker, pip, or uv, with Python 3.13 recommended locally.

Spec sheet

Purpose
PyRIT is an open-source framework for security professionals and engineers to identify risks in generative AI systems.Read 8 Oct 2026 · github.com
Support
The project uses GitHub Issues for bugs, feature requests, and questions, and says support is limited to the resources it lists.Read 8 Oct 2026 · github.com
License
The GitHub repository identifies PyRIT as MIT licensed.Read 8 Oct 2026 · github.com
Human-led interface
CoPyRIT provides a web interface for interacting with AI systems, tracking findings, and collaborating with a team.Read 8 Oct 2026 · microsoft.github.io
Installation
The project documents user installation through Docker or locally with pip or uv, plus contributor Docker and local installation options.Read 8 Oct 2026 · microsoft.github.io
Memory
PyRIT can store conversations, scores, and attack results in SQLite or Azure SQL, and supports exporting results.Read 8 Oct 2026 · microsoft.github.io
Python compatibility
The current documentation recommends Python 3.13 for local installation.Read 8 Oct 2026 · microsoft.github.io
Red teaming
It supports single-turn and multi-turn attack strategies, including Crescendo, TAP, and Skeleton Key.Read 8 Oct 2026 · microsoft.github.io
Scenario evaluations
Its scenario framework supports repeatable evaluations across hundreds of objectives, including content harms, psychosocial risks, and data leakage.Read 8 Oct 2026 · microsoft.github.io
Scoring
Response scoring supports true/false, Likert scale, classification, and custom scorers, including LLM-based and Azure AI Content Safety scoring.Read 8 Oct 2026 · microsoft.github.io
Security reporting
The documentation directs vulnerability reports to the Microsoft Security Response Center and says not to report them through public GitHub issues.Read 8 Oct 2026 · microsoft.github.io
Targets
Documented targets include OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints and WebSockets, and web apps tested with Playwright.Read 8 Oct 2026 · microsoft.github.io
Ways to use
The documentation describes Scanner, GUI, and Framework modes, including command-line assessment through pyrit_scan and pyrit_shell.Read 8 Oct 2026 · microsoft.github.io

Plans and prices

PyRIT Free Open-source framework · install from PyPI or Docker Read 8 Oct 2026 · github.com

Every PyRIT plan and price →

Strong points and limits

Strong points

  • Free MIT-licensed open-source framework
  • Supports single-turn and multi-turn attack strategies
  • Repeatable scenarios across hundreds of objectives
  • Targets major providers and custom endpoints
  • Stores and exports assessment results

Limits

  • Support is limited to listed resources
  • Local installation recommends Python 3.13
  • Vulnerability reports should not use public GitHub issues

Who it suits

Security professionals and engineers evaluating generative AI behavior, safety, and data-leakage risks. It suits teams needing repeatable scenarios, multiple targets, stored results, and human collaboration.

Verdict

PyRIT provides structured red-team, scenario, scoring, and collaboration tools for generative AI assessments. Its free deployment options cover Docker and local workflows, with support handled through project resources and GitHub Issues for general questions.

Where Specifiction places it

Alternatives to PyRITAll alternatives →