garak vs ProofLayer

garak is a command-line scanner for probing language models and systems that use them for security weaknesses. Its checks cover issues such as hallucination, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks. Probes try to elicit failures, while detectors inspect responses for patterns such as unsafe content or a failure to refuse. A standard scan can run without supervision and produce a report, with scan settings available for customization. Integrations include OpenAI, Hugging Face, Cohere, Replicate, AWS Bedrock, LiteLLM, REST-accessible models, and custom Python integrations. Hugging Face models can run locally or be accessed through an inference API or private endpoint. Output includes screen results, a detailed report log, a hit log, and a debug log. garak assesses text output only. It is developed in Linux and OSX and can be installed with pip or from source; listed platforms include Linux, macOS, Windows, and self-hosted use. It is free, with a free plan and no free trial. Listed capabilities include custom metrics, LLM-as-a-judge, and safety evaluations.

ProofLayer scans AI code before deployment, red-teams agents continuously, and protects MCP traffic at runtime, turning findings into audit-ready evidence. Its open-source scanner checks coding agents, MCP servers, prompts, skills, code, and packages from a developer workstation, CI, or as an MCP tool. It flags risks including prompt injection, hallucinated packages, exposed secrets, unsafe MCP tools, and vulnerable generated code. Autonomous attack campaigns test LLM applications, multi-agent systems, RAG pipelines, and MCP servers for issues such as jailbreaks, data exfiltration, tool abuse, RAG poisoning, and memory injection. Verified breaches include replay traces and audit-ready evidence. Runtime protections test MCP traffic for tool poisoning, excessive permissions, unsafe tool execution, and supply-chain risk. Listed targets and integrations include OpenAI, Anthropic, Azure OpenAI, self-hosted Qwen, Llama, and Mistral, plus LangGraph, LangChain, ChromaDB, and MCP servers. ProofLayer says it generates evidence for SOC 2, NIST AI RMF, the EU AI Act, and ISO/IEC 42001. The free Community plan includes a CLI and MCP scanner, 1,700+ detection rules, prompt injection probes, MCP tool testing, a community rule library, and an MIT license.