Arena (formerly Chatbot Arena) vs Promptfoo
Arena (formerly Chatbot Arena)
arena.ai
Promptfoo
promptfoo.devSide by side
| Figure | Arena (formerly Chatbot Arena) | Promptfoo |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | Yes | Yes |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | — | Yes |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 3 | 3 |
| Spec score · LLM Evaluation Tools | 8.8 (#3) | 8.4 (#6) |
| Founded | 2023 | 2024 |
| Funding | Arena says it has $250 million in venture funding led by Felicis and UC Investments, with participation from several other investors. | Crunchbase lists seed and Series A rounds, six investors including Insight Partners and Andreessen Horowitz, and 11–50 employees. |
| Product | Arena is an AI evaluation platform where users compare model responses and vote to shape public leaderboards. | Promptfoo helps developers and enterprises build secure, reliable AI applications. |
| Usage limit | Arena applies a daily usage limit to all users, and usage automatically resets 24 hours after the limit is reached. | The Community plan includes 10k red-team probes per month; the pricing FAQ defines a probe as one request to the target system during red-team testing. |
| What it does | Arena lets people compare anonymous AI-model responses, vote for the better answer, and contribute to public rankings. | Promptfoo is a local-first open-source CLI and library for evaluating and red-teaming LLM applications. |
Plans
Arena (formerly Chatbot Arena)
- Public ArenaFree
- AI evaluationsPriced on request
- AI EvaluationsPriced on request
In their own words
Arena (formerly Chatbot Arena)
Arena, formerly Chatbot Arena, is a free web platform for evaluating AI models by comparing responses and collecting votes for public leaderboards. In Battle mode, it presents two anonymous responses and reveals the model identities after a vote. Arena aggregates qualifying votes with the Bradley-Terry rating system. Side by Side and Direct modes are also available, but only anonymous Battle votes count toward official rankings. Leaderboards cover agent, text, web development, image, video, vision, document and search tasks; comparisons include image generation, image editing and video generation. Leaderboards can be viewed without signing in, while Direct, Side by Side, Image Arena and Video Arena require login. Accounts can use Google or email, and coding Agent Mode can connect to GitHub repositories. Arena applies a daily usage limit that resets 24 hours after it is reached. Video Arena requires login, permits two generations per 24 hours and offers Battle mode only. Arena also provides commercial evaluation services for enterprises, model labs and developers, with results supported by service-level agreements. Conversations may be used in research datasets and private provider evaluations; shared data is de-identified, and Arena advises against submitting sensitive information.
Promptfoo
Promptfoo helps developers and enterprises evaluate prompts and test AI applications for security risks. It compares prompt outputs across multiple models in a web view. Red-team testing simulates users to look for prompt injections, jailbreaks, data leaks, business rule violations and insecure agent tool use. Its getting-started guide lists more than 60 providers, including OpenAI, Anthropic, Google and local models such as Ollama. Integrations include CI systems such as GitHub Actions, GitLab CI, Jenkins, Azure Pipelines, CircleCI and Bitbucket Pipelines, plus Splunk, Burp Suite, Google Sheets and MCP. Installation is available through npm, npx or Homebrew. Promptfoo requires Node.js 22.22.0 or newer and recommends Node.js 24 LTS. The free Community plan includes all LLM evaluation features and 10k red-team probes per month, and can run locally or be self-hosted. Enterprise includes managed cloud deployment; On-Premise runs on customer infrastructure. Promptfoo says it is based in San Francisco, is now part of OpenAI and remains open source.