Skip to content
SPECIFICTION
Get listed

EvoMaster vs Ultra Runner

EvoMaster

github.com

Ultra Runner

github.com

Side by side

FigureEvoMasterUltra Runner
Price fromFree—
Free planYes—
Free trial——
Open source——
WindowsYes—
macOSYes—
LinuxYesYes
APIYes—
Self-hostedYes—
Plans published10
Spec score · Monorepo Management Tools8.2 (#8)6.7 (#33)
InstallationThe recommended install method is PyPI; the project also provides an uber-fat JAR and Docker distribution.The README gives global and project-local installation commands using npm or Yarn.
PurposeEvoMaster automatically generates system-level tests through fuzzing for web and enterprise applications.Ultra Runner is a monorepo script runner and build tool.

Plans

EvoMaster

  • EvoMasterFree

EvoMaster pricing →

Ultra Runner

No plans published.

Ultra Runner pricing →

In their own words

EvoMaster

EvoMaster is a free, open-source tool that uses fuzzing to generate system-level tests for web and enterprise applications. It targets REST, GraphQL, and RPC APIs, including gRPC and Thrift. Black-box mode works with APIs in any programming language, while white-box mode analyzes JVM bytecode. Generated tests can be JUnit 4 or 5 in Java or Kotlin, or suites in Python or JavaScript. More than 30 oracles check issues including HTTP 500 responses, schema mismatches, BOLA, and SQL injection. White-box testing can analyze SQL and MongoDB interactions with PostgreSQL, MySQL, H2, and MongoDB support. Authentication accepts headers and cookies, including tokens or cookies obtained from a login endpoint. REST testing requires OpenAPI 2.0, 3.0, or 3.1. PyPI is the recommended installation route, with Docker and an uber-fat JAR also available. EvoMaster runs on Linux, macOS, Windows, and self-hosted environments, and integrates with GitHub Actions through a custom action.

Visit github.com

Ultra Runner

Ultra Runner has no write-up on Specifiction yet; its figures are above.

Visit github.com