Csmith vs EvoMaster

Csmith is a free, open-source generator of random C programs for finding compiler bugs through differential testing. Its output avoids undefined behavior and includes statistics for each generated program. A documented workflow compiles programs with GCC and Clang, then compares their outputs to identify differences. The tool can also create material for a C test suite when users do not want to write one themselves. Command-line options shown by csmith -h and csmith -hh customize random generation. Csmith runs locally on Linux and Windows. Ubuntu source builds require g++, CMake, and m4, followed by CMake and make; Windows builds are documented with Visual Studio or WSL. The project says it should build on most modern machines with a C++ compiler. Generated programs may loop indefinitely, so execution timeouts are recommended. The repository includes a COPYING license file. Bug reports and suggestions go through GitHub issues, with discussion on a mailing list; maintainers warn replies may be delayed.

EvoMaster is a free, open-source tool that uses fuzzing to generate system-level tests for web and enterprise applications. It targets REST, GraphQL, and RPC APIs, including gRPC and Thrift. Black-box mode works with APIs in any programming language, while white-box mode analyzes JVM bytecode. Generated tests can be JUnit 4 or 5 in Java or Kotlin, or suites in Python or JavaScript. More than 30 oracles check issues including HTTP 500 responses, schema mismatches, BOLA, and SQL injection. White-box testing can analyze SQL and MongoDB interactions with PostgreSQL, MySQL, H2, and MongoDB support. Authentication accepts headers and cookies, including tokens or cookies obtained from a login endpoint. REST testing requires OpenAPI 2.0, 3.0, or 3.1. PyPI is the recommended installation route, with Docker and an uber-fat JAR also available. EvoMaster runs on Linux, macOS, Windows, and self-hosted environments, and integrates with GitHub Actions through a custom action.