Project Moonshot vs PromptGuard

Project Moonshot is an open-source toolkit for evaluating the safety and reliability of large language models and applications. It runs benchmark tests using open-source datasets and metrics, and generates adversarial prompts to probe possible vulnerabilities and misuse. Users can build custom evaluations with their own datasets, optional prompt templates, metrics, and grading scales. Results appear in interactive HTML reports and downloadable raw JSON. Moonshot provides a Web UI, interactive command-line interface, library APIs, and Web APIs, with configurable connections to model providers including OpenAI, Anthropic, Together, and Hugging Face. Users can also create connectors for other models or applications hosted on custom servers. The toolkit implements benchmarks recommended in IMDA’s Starter Kit for safety testing LLM-based applications. Moonshot is free beta software under Apache License 2.0. Installation uses pip and requires Python 3.11; the Web UI additionally requires Node.js 20.11.1 LTS or later, and test assets are needed to run tests. The installation guide recommends Chrome for the Web UI and notes possible installation difficulties on x86 Macs with moonshot-data dependencies.

PromptGuard is a security layer between an application and its LLM provider, inspecting requests before they reach the model. It describes 15 detectors across six layers for risks such as prompt injection, jailbreaks, PII, data exfiltration, toxicity, fraud, secrets, malware and tool injection. The product says it detects and redacts 43 PII types using reversible tokenization. Its SDK can automatically instrument supported LLM calls with one initialization call while leaving existing provider code unchanged. Listed providers include OpenAI, Anthropic, Gemini, Azure, Bedrock, Mistral, Cohere, DeepSeek, Groq, HuggingFace, Ollama and vLLM. Deployment options include managed cloud, hybrid self-hosting and air-gapped installations. In zero-retention mode, prompt and response content is not stored; default security events include a shortened preview and content hash. PromptGuard says customer prompts, completions and documents are not used to train, fine-tune or evaluate models. The Free tier is permanent and includes 20,000 scans a month, one API key, one project and 24-hour log retention. Paid plans include a 14-day money-back guarantee, not a trial.