Skip to content
SPECIFICTION
Get listed

Amazon Bedrock Guardrails vs CyberSecEval

Amazon Bedrock Guardrails

aws.amazon.com

CyberSecEval

meta-llama.github.io

Side by side

FigureAmazon Bedrock GuardrailsCyberSecEval
Price from$0.07/moFree
Free planYesYes
Free trial——
Open source—Yes
WebYesYes
macOS—Yes
Linux—Yes
API—Yes
Self-hosted—Yes
Plans published101
Spec score · LLM Security Tools7.3 (#16)8.9 (#1)
Spec score · AI Guardrail Software7.3 (#14)9.0 (#1)

Plans

Amazon Bedrock Guardrails

  • Content filters — text$0.15 a month
  • Content filters — imageFree
  • Denied topics$0.15 a month
  • Sensitive information filters$0.10 a month
  • Sensitive information filters — regular expressionFree
  • Word filtersFree
  • Contextual grounding checks$0.10 a month
  • InvokeGuardrailChecks — content filters$0.07 a month
  • InvokeGuardrailChecks — prompt attack$0.08 a month
  • InvokeGuardrailChecks — sensitive information filters$0.10 a month

Amazon Bedrock Guardrails pricing →

CyberSecEval

  • CyberSecEval 4Free

CyberSecEval pricing →

In their own words

Amazon Bedrock Guardrails

Amazon Bedrock Guardrails is a cloud service for applying checks to model interactions. Listed capabilities include prompt injection defense, output guardrails, PII redaction, and secrets detection. Controls include text and image content filters, denied-topic rules, sensitive-information filters, word filters, and contextual grounding checks. Separate InvokeGuardrailChecks options cover content filters, prompt attacks, and sensitive-information filters; content-filter and prompt-attack checks are specified for text only. The service is web-based and has no free plan. Listed prices include $0.15 a month for text content filters on Standard and Classic tiers, and $0.15 a month for denied topics. Sensitive-information filters and contextual grounding checks each cost $0.10 a month. Image content filters, regular-expression sensitive-information filters, and word filters are listed at free. InvokeGuardrailChecks content filters cost $0.07 a month, prompt-attack checks cost $0.08 a month, and sensitive-information checks cost $0.10 a month.

Visit aws.amazon.com

CyberSecEval

CyberSecEval 4 is a benchmark suite for assessing cybersecurity vulnerabilities and defensive capabilities in large language models. Its tests measure model responses to cyberattack requests and whether borderline benign requests are incorrectly refused. Secure-code checks look for insecure suggestions in both instruction and autocomplete settings. Prompt-injection tests cover textual, multilingual textual, and visual attacks; a judge LLM evaluates whether injected instructions succeeded. The code-interpreter benchmark evaluates compliance with malicious prompts and sorts judged responses into three categories: extremely malicious, potentially malicious, or non-malicious. AutoPatchBench measures patch-generation agents on fuzzed C and C++ vulnerabilities, while CyberSOCEval, developed with CrowdStrike, covers malware analysis and threat-intelligence reasoning. The guide lists API support for OpenAI, Anyscale, and Together, and explains adding custom providers for self-hosted models. Setup uses a Python virtual environment, installed requirements, and a Python command-line module. The suite is free under the MIT license for research and commercial use. AutoPatch needs substantial resources: the page recommends at least 80 CPUs and 8 TB of storage.

Visit meta-llama.github.io