Project Moonshot vs SuperRed
Project Moonshot
aiverifyfoundation.sgSuperRed
rdi.berkeley.eduSide by side
| Figure | Project Moonshot | SuperRed |
|---|---|---|
| Price from | Free | — |
| Free plan | Yes | — |
| Free trial | — | — |
| Open source | — | — |
| Web | Yes | — |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | Yes |
| API | Yes | — |
| Self-hosted | Yes | — |
| Plans published | 1 | 0 |
| Spec score · AI Security Testing Tools | 8.3 (#8) | 6.7 (#27) |
| Installation | The maker's instructions install Moonshot with pip and run the web UI locally at localhost:3000. | The guide installs the framework with pip install superred and describes it as a Python framework whose guide assumes familiarity with Python and asyncio. |
| Intended users | The maker describes Moonshot as a tool for AI developers, compliance teams, and AI system owners evaluating LLMs and LLM applications. | The project describes use by red-teamers, system builders, and evaluators, and its guide covers wrapping systems, writing attackers, defining success criteria, and running evaluations. |
| License and maturity | The repository identifies Moonshot as beta software released under the Apache Software License 2.0. | PyPI lists the package under the MIT license and classifies its development status as Alpha. |
| Maker | AI Verify Foundation is a not-for-profit wholly owned subsidiary of Singapore’s Infocommunications Media Development Authority. | The site says SuperRed was made at the University of California, Berkeley. |
| Purpose | Project Moonshot is an open-source toolkit for testing the safety and reliability of LLMs and LLM applications through benchmarking and red teaming. | SuperRed is an open-source framework for red-teaming chatbots, agents, and assistants by testing whether attacks can violate security properties. |
| Requirements | Moonshot requires Python 3.11, and its web UI requires Node.js 20.11.1 LTS or above and npm 10.8.0 or above. | The package requires Python 3.11 through 3.13; Python 3.14 is not supported. |
Plans
In their own words
Project Moonshot
Project Moonshot is an open-source toolkit for evaluating the safety and reliability of large language models and applications. It runs benchmark tests using open-source datasets and metrics, and generates adversarial prompts to probe possible vulnerabilities and misuse. Users can build custom evaluations with their own datasets, optional prompt templates, metrics, and grading scales. Results appear in interactive HTML reports and downloadable raw JSON. Moonshot provides a Web UI, interactive command-line interface, library APIs, and Web APIs, with configurable connections to model providers including OpenAI, Anthropic, Together, and Hugging Face. Users can also create connectors for other models or applications hosted on custom servers. The toolkit implements benchmarks recommended in IMDA’s Starter Kit for safety testing LLM-based applications. Moonshot is free beta software under Apache License 2.0. Installation uses pip and requires Python 3.11; the Web UI additionally requires Node.js 20.11.1 LTS or later, and test assets are needed to run tests. The installation guide recommends Chrome for the Web UI and notes possible installation difficulties on x86 Macs with moonshot-data dependencies.
SuperRed
SuperRed has no write-up on Specifiction yet; its figures are above.