PyRIT

Overview

PyRIT is ranked #7 of 29 in AI security testing tools on Specifiction. It runs on API, Linux, macOS, Self-hosted, Web, Windows. There is a free plan.

Compared on AI security testing tools

Free plan

Yesazure.github.io

Prompt injection tests

Yesazure.github.io

Jailbreak tests

Yesazure.github.io

Data leakage tests

Yesazure.github.io

Unsafe output tests

Yesazure.github.io

Custom test cases

Yesazure.github.io

Deployment mode

self_hostedazure.github.io

Facts

Purpose

PyRIT is an open-source framework for automated and human-led red teaming to assess the security and safety of generative AI systems.microsoft.github.io · 30 Sept 2026

Attack strategies

It supports single-turn and multi-turn strategies including Crescendo, TAP, and Skeleton Key.microsoft.github.io · 30 Sept 2026

Scenarios

Its scenarios package attack strategies and datasets for repeatable assessments of content harms, psychosocial risks, data leakage, and other objectives.microsoft.github.io · 30 Sept 2026

Interfaces

Users can run assessments with the command-line scanner and interactive shell, use the CoPyRIT graphical interface, or build workflows with the framework.microsoft.github.io · 30 Sept 2026

Targets

Documented targets include OpenAI, Azure, Anthropic, Google, Hugging Face, custom HTTP endpoints and WebSockets, and web apps tested with Playwright.microsoft.github.io · 30 Sept 2026

Components

The modular framework includes targets, converters, scorers, memory, datasets, attacks, and scenarios.microsoft.github.io · 30 Sept 2026

Prompt conversion

Converters transform prompts through text operations such as encoding, obfuscation, translation, and semantic changes, as well as conversions among text, images, audio, video, and files.microsoft.github.io · 30 Sept 2026

Scoring

Scorers can return true/false or normalized 0.0–1.0 scores and can use LLMs, Azure AI Content Safety, or custom logic.microsoft.github.io · 30 Sept 2026

Memory

Built-in memory can track conversations, scores, and attack results using SQLite or Azure SQL.microsoft.github.io · 30 Sept 2026

Security

PyRIT recommends Azure Key Vault for shared or deployed configuration and warns that plaintext .env files are less secure.microsoft.github.io · 30 Sept 2026

Credential handling

In CoPyRIT, an API key entered when creating a target is stored in memory only and is not persisted to disk.microsoft.github.io · 30 Sept 2026

Installation

The documentation provides local installation with pip or uv and separate Docker installation options.microsoft.github.io · 30 Sept 2026

Compatibility limit

The local installation page lists Python 3.10 through 3.14 as prerequisites, and the contributor setup page requires Node.js 22 or higher for the frontend.microsoft.github.io · 30 Sept 2026

Where it ranks on Specifiction

• Best AI Security Testing Tools in 2026#7 of 29

Sources

• microsoft.github.io/PyRIT/latest/· checked 30 Sept 2026
• microsoft.github.io/PyRIT/latest/code/framework/· checked 30 Sept 2026
• microsoft.github.io/PyRIT/latest/code/converters/converters· checked 30 Sept 2026
• microsoft.github.io/PyRIT/latest/getting-started/pyrit-conf· checked 30 Sept 2026
• microsoft.github.io/PyRIT/latest/gui/gui/· checked 30 Sept 2026
• microsoft.github.io/PyRIT/latest/getting-started/install-lo· checked 30 Sept 2026