Augustus vs LLMMap

Augustus is a free, self-hosted Go scanner for security professionals assessing large language models. It runs adversarial probes and produces vulnerability reports, with more than 210 probes across 47 attack categories, including jailbreaks, prompt injection, data extraction, safety benchmarks, and agent attacks. It supports 28 provider categories and 43 generator variants, including hosted providers, Ollama, and custom REST endpoints. More than 90 detectors cover approaches such as pattern matching, LLM-as-a-judge, HarmJudge, Perspective API, and unsafe-content detection; seven prompt transformations include encoding, paraphrase, poetry, low-resource-language translation, and case changes. Results can be written as tables, JSON, JSONL, or HTML. The project lists concurrent scanning, rate limits, retries, and timeout handling. Augustus compiles to a portable single binary and can be installed with Go or built from source; the README requires Go 1.27.0 or later. It is Apache 2.0 licensed. The project is intended for authorized security testing, and users are told to test only systems they own or have permission to assess. Provider API use may require credentials.

LLMMap has no write-up on Specifiction yet; its figures are above.