Skip to content
SPECIFICTION
Get listed

Promptfoo

promptfoo.dev

Free plan Open source WebWindowsmacOSLinuxAPISelf-hosted
Visit promptfoo.dev
Spec score 8.6 #4 of 27 in AI Red Teaming Tools

What it is

Promptfoo helps developers and enterprises evaluate prompts and test AI applications for security risks. It compares prompt outputs across multiple models in a web view. Red-team testing simulates users to look for prompt injections, jailbreaks, data leaks, business rule violations and insecure agent tool use. Its getting-started guide lists more than 60 providers, including OpenAI, Anthropic, Google and local models such as Ollama. Integrations include CI systems such as GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI and Bitbucket Pipelines, plus Splunk, Burp Suite, Google Sheets and MCP. Installation is available through npm, npx or Homebrew. Promptfoo requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS. The free Community plan includes all LLM evaluation features and 10k red-team probes per month, and can run locally or be self-hosted. Enterprise includes managed cloud deployment; On-Premise runs on customer infrastructure. Promptfoo says it is based in San Francisco, is now part of OpenAI and remains open source.

Spec sheet

What it does
Promptfoo is a local-first open-source CLI and library for evaluating and red-teaming LLM applications.Read 15 Oct 2026 · promptfoo.dev
Product
Promptfoo helps developers and enterprises build secure, reliable AI applications.Read 3 Oct 2026 · promptfoo.dev
Deployment
The Community plan can run locally or be self-hosted, while Enterprise includes managed cloud deployment and On-Premise supports deployment on the customer’s infrastructure.Read 3 Oct 2026 · promptfoo.dev
Integrations
Documented integrations include LangChain, LangGraph, CrewAI, MCP, Splunk, Burp Suite, GitHub Actions, GitLab, Jenkins, CircleCI, Azure Pipelines, and Bitbucket Pipelines.Read 15 Oct 2026 · promptfoo.dev
Security
The site displays SOC 2 and ISO 27001 certification badges.Read 3 Oct 2026 · promptfoo.dev
Support
Community support is provided through chat and GitHub issues, while Enterprise includes professional services and dedicated Slack support.Read 15 Oct 2026 · promptfoo.dev
License
The open-source project is distributed under the MIT license.Read 15 Oct 2026 · github.com
Headquarters
Promptfoo identifies San Francisco, California as its base and headquarters.Read 15 Oct 2026 · promptfoo.dev
Founded
2024Read 23 Sep 2026 · promptfoo.dev
Company
Promptfoo says it is based in San Francisco, California, and is now part of OpenAI while remaining open source.Read 3 Oct 2026 · promptfoo.dev
Acquisition news
Promptfoo announced on March 9, 2026 that it had agreed to be acquired by OpenAI while remaining open source.Read 15 Oct 2026 · promptfoo.dev
Browser testing
An optional Playwright-based browser provider supports Chromium, Edge, Firefox, and WebKit interactions.Read 15 Oct 2026 · promptfoo.dev
CI/CD
Promptfoo can run evaluations and security checks in CI/CD pipelines using configuration files and provider credentials.Read 15 Oct 2026 · promptfoo.dev
Cloud viewer
The browser viewer is free at promptfoo.app and supports private organization links plus self-hosted sharing endpoints.Read 15 Oct 2026 · promptfoo.dev
Company founding
Promptfoo's co-founder said the company was founded in 2024.Read 15 Oct 2026 · promptfoo.dev
Company legal name
Crunchbase lists the legal name as Promptfoo, Inc.Read 15 Oct 2026 · crunchbase.com
Company size
LinkedIn lists Promptfoo as a privately held software-development company with 11–50 employees.Read 15 Oct 2026 · linkedin.com
Compliance testing
Its GDPR preset tests privacy, access control, personal-data exposure, deletion flows, and automated-decision risks, but the documentation says it is not legal certification.Read 15 Oct 2026 · promptfoo.dev
Custom extensibility
Developers can write JavaScript, TypeScript, Python, Ruby, or script-based custom providers for APIs not built in.Read 15 Oct 2026 · promptfoo.dev
Documentation depth
The documentation includes installation, evaluations, assertions, runtime, red teaming, providers, integrations, code scanning, model audit, enterprise, API references, and release notes.Read 15 Oct 2026 · promptfoo.dev
Evaluation
It tests prompts across multiple providers, grades outputs with assertions, and calculates accuracy, safety, and performance metrics.Read 15 Oct 2026 · promptfoo.dev
Evaluations
Promptfoo evaluates prompts across multiple models and opens a web view for comparing outputs.Read 3 Oct 2026 · promptfoo.dev
Funding
Crunchbase lists seed and Series A rounds, six investors including Insight Partners and Andreessen Horowitz, and 11–50 employees.Read 15 Oct 2026 · crunchbase.com
HTTP API support
The HTTP provider can call arbitrary endpoints, build requests from templates, transform responses, authenticate, and handle tool-calling formats.Read 15 Oct 2026 · promptfoo.dev
Independent package evidence
Homebrew lists promptfoo as MIT licensed and provides macOS Apple Silicon plus Linux ARM64 and x86_64 bottles.Read 15 Oct 2026 · formulae.brew.sh
Installation
Installation is available through npm, npx, Homebrew, and a project dependency; npm and npx require Node.js 22.22.0 or newer.Read 15 Oct 2026 · promptfoo.dev
Limit: community red teaming
The Community plan limits red teaming to 10,000 probes per month, while Enterprise offers custom limits.Read 15 Oct 2026 · promptfoo.dev
Limit: independent reviews
G2 says there are not enough reviews to provide buying insight; Capterra and GetApp also show no user reviews.Read 15 Oct 2026 · g2.com
Limit: pricing transparency
Enterprise and On-Premise pricing is customized and requires contacting Promptfoo for a quote.Read 15 Oct 2026 · promptfoo.dev
Model providers
Provider documentation covers OpenAI, Anthropic, Google, Vertex, HTTP, JavaScript, Python, Ruby, shell, OpenAI Agents, and other services.Read 15 Oct 2026 · promptfoo.dev
Node API
The Node API exports evaluation, provider, assertion, cache, guardrail, and TypeScript type interfaces, with stable and experimental API areas identified.Read 15 Oct 2026 · promptfoo.dev
Praise: independent reviews
G2, Capterra, and GetApp pages each show zero user reviews, so no repeated reviewer praise is available.Read 15 Oct 2026 · g2.com
Providers
The getting-started guide says Promptfoo supports 60+ providers, including OpenAI, Anthropic, Google, and local models such as Ollama.Read 3 Oct 2026 · promptfoo.dev
Rating on Capterra
0.0 out of 5 from 0 reviews.Read 15 Oct 2026 · capterra.com
Rating on G2
0 out of 5 from 0 reviews.Read 15 Oct 2026 · g2.com
Rating on GetApp
0 out of 5 from 0 reviews.Read 15 Oct 2026 · getapp.com
Red teaming
Its red-team workflow generates adversarial tests for vulnerabilities before deployment and maps testing to OWASP and NIST frameworks.Read 15 Oct 2026 · promptfoo.dev
Red-team plugins
The project documents 157 plugins in six categories, including security, compliance, datasets, trust and safety, brand, and custom tests.Read 15 Oct 2026 · promptfoo.dev
Release activity
January 2026 release notes list adaptive rate limiting, Transformers.js local inference, telecom red-team plugins, video-generation providers, and a VS Code extension.Read 15 Oct 2026 · promptfoo.dev
Runtime requirement
Promptfoo requires Node.js 22.22.0 or newer, and recommends Node.js 24 LTS.Read 3 Oct 2026 · promptfoo.dev
Security caveat
Promptfoo states that custom code, providers, transforms, hooks, plugins, and templates execute without sandboxing under the user's permissions.Read 15 Oct 2026 · gitblind.noratr.app
Security certifications
Promptfoo announced SOC 2 Type II compliance and ISO 27001 certification on July 11, 2025.Read 15 Oct 2026 · promptfoo.dev
Self-hosting
Community runs locally, while Enterprise On-Premise provides self-hosted deployment, network isolation, dedicated runners, and customer-managed infrastructure.Read 15 Oct 2026 · promptfoo.dev
System requirements
The documentation recommends Node.js 24 LTS and supports Windows, macOS, and Linux command-line usage.Read 15 Oct 2026 · promptfoo.dev
Usage limit
The Community plan includes 10k red-team probes per month; the pricing FAQ defines a probe as one request to the target system during red-team testing.Read 3 Oct 2026 · promptfoo.dev

Plans and prices

Community Free Free Forever 10k red-team probes/month · local or self-hosted · community support Read 15 Oct 2026 · promptfoo.dev
Enterprise Priced on request Custom custom red-team limits · team collaboration · continuous monitoring · SSO/RBAC · API access · managed cloud · priority support and SLA Read 15 Oct 2026 · promptfoo.dev
On-Premise Priced on request Custom enterprise features · customer infrastructure · complete data isolation · dedicated runner · deployment engineer Read 15 Oct 2026 · promptfoo.dev

Every Promptfoo plan and price →

Strong points and limits

Strong points

  • Compares prompt outputs across multiple models.
  • Red-team testing targets several classes of AI risk.
  • Lists more than 60 model providers.
  • Community plan includes 10k red-team probes per month.
  • Integrates with CI systems and other listed tools.

Limits

  • Requires Node.js 22.22.0 or newer.
  • Enterprise and On-Premise pricing is custom.
  • Community plan includes a monthly probe limit.

Who it suits

It suits developers and enterprises evaluating prompts or testing AI applications for risks. Teams can choose local or self-hosted use, managed cloud deployment, or deployment on their own infrastructure.

Promptfoo in depth

Read the full write-up

Promptfoo is an evaluation and red-teaming platform for developers building AI applications. It suits teams that want to compare model outputs and probe for security risks in the same workflow. Its free Community plan covers substantial evaluation needs, but the monthly probe cap and Node.js requirement may shape the choice.

Overview

Promptfoo brings prompt evaluation and red-team testing together, with support for hosted and local models. Founded in 2024 and based in San Francisco, it says it is now part of OpenAI while remaining open source. Community runs locally or self-hosted; Enterprise adds managed cloud deployment, and On-Premise runs on the customer’s infrastructure.

Key features

Evaluate prompts across models

Promptfoo evaluates prompts across multiple models and provides a web view for comparing outputs. Support for 60+ providers includes OpenAI, Anthropic, Google, and local models such as Ollama. That range is useful when a team needs to compare providers in one evaluation workflow rather than commit to a single model ecosystem.

Community includes all LLM evaluation features, custom metrics, LLM-as-a-judge, safety evaluations, human review workflows, and CI/CD integration. Integrations include GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI, Bitbucket Pipelines, Splunk, Burp Suite, Google Sheets, and MCP. This makes the free plan capable of supporting development and security workflows; teams should confirm that its specific integrations match their tooling.

Probe for risks

Red-team runs simulate users to look for prompt injections, jailbreaks, data leaks, business-rule violations, and insecure agent tool use. Community allows 10k probes per month, with each probe counting as one request to the target system. That is a meaningful allowance for smaller-scale or periodic testing, but frequent testing across multiple targets can make the cap a constraint. Enterprise offers custom red-teaming limits and continuous monitoring for teams needing more capacity or ongoing checks.

Installation and security

Promptfoo can be installed with npm, npx, or Homebrew. It requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS, a notable prerequisite for teams standardizing on an older runtime. Homebrew supports Mac and Linux. The site displays SOC 2 and ISO 27001 certification badges.

Pricing

Community: free. This plan includes all LLM evaluation features, all model providers and integrations, local or self-hosted deployment, community support, and 10k red-team probes per month. It is the clear fit for individual developers and teams whose testing stays within that allowance and who do not need enterprise support or managed deployment.

Enterprise: custom pricing. Pricing is based on team size and needs. The plan adds custom red-teaming limits, team sharing, continuous monitoring, SSO, API access, managed cloud deployment, priority support, and SLA guarantees. It suits organizations that need shared workflows, ongoing monitoring, or a managed service; the trade-off is a custom-priced commitment rather than a published rate.

On-Premise: custom pricing. It includes Enterprise features plus deployment on the customer’s infrastructure, complete data isolation, a dedicated runner, and an assigned deployment engineer. This is the appropriate option where infrastructure control and isolation are central requirements, but it carries custom pricing.

Platforms

Promptfoo supports API, Linux, macOS, self-hosted, web, and Windows. Its deployment choices span local and self-hosted use through Community, managed cloud with Enterprise, and customer-infrastructure deployment with On-Premise. The broad range accommodates different operating and hosting preferences, though installation requires the stated Node.js version.

Who it's for

Promptfoo is a strong fit for developers and AI teams that need both model evaluation and adversarial testing, particularly when they use several hosted or local providers and want evaluation in CI/CD. Community is suited to teams comfortable with local or self-hosted operation and a fixed probe allowance. Organizations needing SSO, continuous monitoring, SLAs, managed cloud, or full infrastructure isolation should consider a paid plan. It is less suitable for teams unable to meet the Node.js requirement or whose red-team volume exceeds Community’s monthly cap without custom limits.

Pros and cons

Broad evaluation coverage at no cost: Community includes all evaluation features, providers, and integrations, alongside custom metrics, safety evaluation, and human review workflows.

Evaluation and security testing in one product: Teams can compare model outputs and probe for risks including insecure agent tool use without separating those tasks across tools.

Flexible deployment: Local, self-hosted, managed cloud, and customer-infrastructure options serve different operational requirements.

Probe cap on the free plan: Community’s 10k monthly probes may limit frequent or broad red-team campaigns.

Runtime prerequisite: Node.js 22.22.0 or newer may add setup work for teams on older versions.

Custom-priced higher tiers: Enterprise and On-Premise provide organizational controls and deployment options, but pricing is not a fixed published amount.

Alternatives

DeepEval is a free, Apache 2.0-licensed open-source LLM evaluation framework with local and CI/CD test runners; choose it when evaluation is the priority and a standalone framework is enough.

Maxim AI offers a free Developer plan for up to 3 seats, one workspace, 10k logs per month, and three-day data retention; it may suit a small team seeking a bounded hosted workspace.

Whisper provides MIT-licensed open-source code and model weights for local installation with Python and ffmpeg; choose it when that package is what you need.

Giskard offers a free open-source library for local deployment, with a basic LLM vulnerability scan and RAG evaluation report; it is an option for teams focused on those checks.

Braintrust has a free Starter plan with 1 GB processed data, 10,000 scores, 14-day retention, and unlimited users, projects, and datasets; choose it when those evaluation allowances fit your workflow.

Galileo offers a Pro plan at $100 a month, billed yearly, with 50,000 traces per month, standard RBAC, advanced analytics, and dedicated Slack support; it may fit teams seeking those paid analytics and support provisions.

garak is a free open-source tool; consider it as an alternative if you want a free tool.

Parea AI offers a free plan at free with up to two team members, 3k logs per month, one-month retention, and 10 deployed prompts; it may suit small teams whose needs match those limits.

For a broader comparison, see LLM Evaluation Tools, AI Prompt Management Software, AI Red Teaming Tools, AI Security Testing Tools, AI LLM Evaluation Tools, and AI Agent Evaluation Tools.

Verdict

Choose Promptfoo if you want model evaluation and red-team testing together, broad provider support, and local or self-hosted use without a software charge. Its main appeal is the breadth of Community; its main caveats are the 10k monthly probe limit, the Node.js prerequisite, and custom pricing for enterprise deployment and controls. Teams that need more probes or organizational support should request Enterprise pricing, while infrastructure isolation points to On-Premise.

Verdict

Promptfoo combines prompt evaluation, red-team testing and multiple deployment choices. Check the Node.js requirement and probe allowance, and request pricing for Enterprise or On-Premise.

Where Specifiction places it

Alternatives to PromptfooAll alternatives →