Promptfoo vs RedLens AI

Promptfoo helps developers and enterprises evaluate prompts and test AI applications for security risks. It compares prompt outputs across multiple models in a web view. Red-team testing simulates users to look for prompt injections, jailbreaks, data leaks, business rule violations and insecure agent tool use. Its getting-started guide lists more than 60 providers, including OpenAI, Anthropic, Google and local models such as Ollama. Integrations include CI systems such as GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI and Bitbucket Pipelines, plus Splunk, Burp Suite, Google Sheets and MCP. Installation is available through npm, npx or Homebrew. Promptfoo requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS. The free Community plan includes all LLM evaluation features and 10k red-team probes per month, and can run locally or be self-hosted. Enterprise includes managed cloud deployment; On-Premise runs on customer infrastructure. Promptfoo says it is based in San Francisco, is now part of OpenAI and remains open source.

RedLens AI runs automated adversarial attacks against production AI systems and maps findings to controls used in security audits. It is aimed at healthcare, law enforcement and regulated enterprise organizations deploying AI. Its Master Attack Schema v2.0 covers six attack categories and 22 methodologies, with findings scored by CVSS. Each assessment produces a PDF containing a risk score, vulnerabilities, remediation steps and mappings to frameworks including HIPAA, NIST AI RMF, OWASP GenAI Top 10, MITRE ATLAS, ISO/IEC 42001, FBI CJIS Security Policy and AIUC-1. The mappings are evidence, not certification. Retests can run daily or weekly to detect drift, and critical findings can trigger logged email alerts. A Python CLI and GitHub Action support CI/CD scans with severity gates and SARIF output; findings can stream to listed SIEM services or an HMAC-signed webhook. Deployment options include hosted SaaS and Enterprise self-hosted Docker. In the self-hosted package, stored data stays in the customer's Postgres, but attack generation and judging still call Anthropic's API using the customer's key. The Free plan includes 25 hosted scans monthly; paid plans start at $199.00 per month, while Developer starts at $799.00 per month.