Specifiction
For vendors
AdvertiseTop spots, featured rows, home sponsor Get listedAdd a product we don't cover yet Claim a listingFree: verify the domain, then correct your page Vendor dashboardNumbers, orders and placements
List your tool
Vendor dashboard
Tech riderRev. 4 Oct 2026

RAGChecker

Ch 246.5of 30 AI LLM Evaluation Tools
  1. 1Runs onself-hosted
  2. 2CostsFree plan
  3. 3Deploymentself-hosted
3 lines stated Written from the maker's own pages: github.com
Visit site Alternatives
The RAGChecker homepage

Overview

RAGChecker is ranked #24 of 30 in AI LLM evaluation tools on Specifiction. It runs on Self-hosted. There is a free plan.

RAGChecker plans and pricing

All plans
RAGChecker Free Apache-2.0 licensed open-source framework · installable with pip github.com · 4 Oct 2026

Compared on AI LLM evaluation tools

Deployment
self-hostedgithub.com

Facts

Purpose
RAGChecker is an automatic evaluation framework for assessing and diagnosing retrieval-augmented generation systems.github.com · 4 Oct 2026
Pipeline metrics
It provides overall metrics for the RAG pipeline and diagnostic metrics for retrieval and generation components.github.com · 4 Oct 2026
Fine-grained evaluation
Its fine-grained evaluation uses claim-level entailment operations.github.com · 4 Oct 2026
Retriever metrics
Documented retriever metrics include claim recall and context precision.github.com · 4 Oct 2026
Generator metrics
Documented generator metrics include context utilization, noise sensitivity, hallucination, self-knowledge, and faithfulness.github.com · 4 Oct 2026
Benchmark
The repository says its benchmark was released and links a guide for using it.github.com · 4 Oct 2026
Input requirement
For each query, the only required annotation is the ground-truth answer; generated responses and retrieved context are also accepted in the input format.github.com · 4 Oct 2026
Installation
The maker documents installation with pip install ragchecker and requires downloading the spaCy en_core_web_sm model in its quick start.github.com · 4 Oct 2026
Usage
The framework can be run through a CLI or Python API, with selectable metric groups or all metrics.github.com · 4 Oct 2026
Model setup
The example pipeline uses configurable extractor and checker models and documents an AWS Bedrock Llama 3 70B example.github.com · 4 Oct 2026
Integration
The README documents an integration with LlamaIndex for evaluating RAG applications built with it.github.com · 4 Oct 2026
License
The project is licensed under Apache-2.0.github.com · 4 Oct 2026
Security reporting
The contribution guide asks users to report potential security issues to AWS/Amazon Security through its vulnerability reporting page rather than public GitHub issues.github.com · 4 Oct 2026
Support and contributions
The project directs bug reports and feature requests to its GitHub issue tracker and welcomes pull requests.github.com · 4 Oct 2026

Best RAGChecker alternatives

See all 20
All accessCh 01 Weights & Biases Free planFree trialAPI from $60/mo8.0 All accessCh 02 Evidently AI Free planFree trialAPI from $80/mo7.8 All accessCh 03 Vellum Free planAndroidBrowser from $30/mo7.7 All accessCh 04 Opik Free planFree trialAPI from $19/mo7.5 All accessCh 05 Promptfoo Free planAPILinux Free to start7.4 All accessCh 06 Maxim AI Free planFree trialAPI from $29/mo7.3

Where it ranks on Specifiction

Is RAGChecker yours?

Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.

Claim it free Promote RAGChecker

Sources