Tech riderRev. 20 Sept 2026
EvalPlus
- 1Runs onAPI, Linux, self-hosted
- 2CostsNot stated by the maker
- 3Deployment optionsself-hosted
2 lines stated Written from the maker's own pages: github.com
Overview
EvalPlus is ranked #15 of 29 in LLM evaluation tools on Specifiction. It runs on API, Linux, Self-hosted.
Compared on LLM evaluation tools
- Deployment options
- self-hostedgithub.com
Facts
- Purpose
- EvalPlus is an evaluation framework for assessing code generated by large language models.github.com · 5 Oct 2026
- HumanEval+
- HumanEval+ expands the original HumanEval benchmark with 80 times more tests.github.com · 5 Oct 2026
- MBPP+
- MBPP+ expands the original MBPP benchmark with 35 times more tests.github.com · 5 Oct 2026
- Efficiency evaluation
- EvalPerf evaluates the efficiency of LLM-generated code using performance-exercising tasks and test inputs.github.com · 5 Oct 2026
- Leaderboard
- The EvalPlus site provides leaderboards for HumanEval+ and MBPP+ evaluation results.evalplus.github.io · 5 Oct 2026
- EvalPerf leaderboard
- The EvalPlus site provides a leaderboard for EvalPerf code efficiency results.evalplus.github.io · 5 Oct 2026
- RepoQA
- The EvalPlus site describes RepoQA as a project to design evaluators for long-context code understanding.evalplus.github.io · 5 Oct 2026
- Installation
- The README documents installation through pip, including optional vLLM and performance extras.github.com · 5 Oct 2026
- Docker
- The README documents running code evaluation in Docker.github.com · 5 Oct 2026
- Model backends
- Documented backends include Hugging Face Transformers, vLLM, OpenAI-compatible servers, OpenAI, Anthropic, Google, Amazon Bedrock, Ollama, and Intel Gaudi.github.com · 5 Oct 2026
- Security
- The README describes Docker-based code execution as safe code execution.github.com · 5 Oct 2026
- Platform limit
- The README labels EvalPerf code efficiency evaluation as *nix only.github.com · 5 Oct 2026
- License
- The repository is distributed under the Apache License, Version 2.0; the license notes that files under evalplus/eval additionally comply with the MIT License.github.com · 5 Oct 2026
- Intended audience
- The project is intended for evaluating LLM performance on code-related tasks.evalplus.github.io · 5 Oct 2026
Best EvalPlus alternatives
See all 20 All accessCh 01 Promptfoo Free planAPILinux Free to start7.5 All accessCh 02 DeepEval Free planLinuxMac Free to start7.4 All accessCh 03 Maxim AI Free planFree trialAPI from $29/mo7.4 All accessCh 04 Giskard Free planAPILinux Free to start7.2 All accessCh 05 Braintrust Free planAPIself-hosted from $249/mo7.1 All accessCh 06 Galileo Free planAPIself-hosted from $100/mo7.1
Where it ranks on Specifiction
- Best LLM Evaluation Tools in 2026#15 of 29
Is EvalPlus yours?
Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.
Sources
- github.com/evalplus/evalplus· checked 5 Oct 2026
- evalplus.github.io· checked 5 Oct 2026
- github.com/evalplus/evalplus/blob/master/LICENSE· checked 5 Oct 2026