Skip to content
SPECIFICTION
Get listed

Galileo vs PyRIT

Galileo

galileo.ai

PyRIT

microsoft.github.io

Side by side

FigureGalileoPyRIT
Price from$100/moFree
Free plan—Yes
Free trial——
Open source——
WebYesYes
Linux—Yes
APIYesYes
Self-hostedYesYes
Plans published31
Spec score · LLM Evaluation Tools7.8 (#11)7.5 (#18)
PurposeGalileo is an observability, evaluation, and guardrail platform for GenAI and agentic applications.PyRIT is an open-source framework for security professionals and engineers to identify risks in generative AI systems.
SupportThe Pro plan includes dedicated Slack support, and Enterprise includes 24/7 support via Slack, email, or phone.The project uses GitHub Issues for bugs, feature requests, and questions, and says support is limited to the resources it lists.

Plans

Galileo

  • Pro$100 a month
  • EnterprisePriced on request
  • FreePrice not published

Galileo pricing →

PyRIT

  • PyRITFree

PyRIT pricing →

In their own words

Galileo

Galileo is an observability, evaluation and guardrail platform for generative AI and agentic applications. It provides ready-made evaluations for RAG, agents, safety and security, as well as custom evaluators. Observability tools analyze agent behavior to surface patterns and failure modes, while Galileo Experiments let teams evaluate datasets and prompts systematically and compare results. Evaluation scores can guide agent actions, tool access and escalation paths. Developers can use Python and TypeScript SDKs or the API, with integrations including CrewAI, Google ADK, LangChain and LangGraph, OpenAI, Microsoft Agent Framework, Strands Agents and Vercel AI SDK. Deployment options include SaaS, virtual private cloud and on-premises. The Free plan includes 5,000 traces per month, unlimited users and unlimited custom evaluations. Pro costs $100 a month, billed yearly, and includes 50,000 traces per month; pricing scales with trace volume. SSO supports OIDC and SAML only on a customer's own Galileo cluster, not the free version or hosted app.galileo.ai service.

Visit galileo.ai

PyRIT

PyRIT is a free, MIT-licensed framework for security professionals and engineers assessing risks in generative AI systems. It supports single-turn and multi-turn red-team strategies such as Crescendo, TAP, and Skeleton Key. Scenario evaluations can be repeated across hundreds of objectives, including content harms, psychosocial risks, and data leakage. CoPyRIT provides a web interface for interacting with AI systems, recording findings, and collaborating with a team. Documented targets include OpenAI, Azure, Anthropic, Google, HuggingFace, custom HTTP endpoints and WebSockets, and web apps tested with Playwright. PyRIT stores conversations, scores, and attack results in SQLite or Azure SQL and can export results. Scoring supports true/false, Likert, classification, and custom scorers, including LLM-based and Azure AI Content Safety methods. Scanner, GUI, Framework, pyrit_scan, and pyrit_shell modes are documented. Installation is available through Docker, pip, or uv, with Python 3.13 recommended locally.

Visit microsoft.github.io