Csmith vs Schemathesis

Csmith is a free, open-source generator of random C programs for finding compiler bugs through differential testing. Its output avoids undefined behavior and includes statistics for each generated program. A documented workflow compiles programs with GCC and Clang, then compares their outputs to identify differences. The tool can also create material for a C test suite when users do not want to write one themselves. Command-line options shown by csmith -h and csmith -hh customize random generation. Csmith runs locally on Linux and Windows. Ubuntu source builds require g++, CMake, and m4, followed by CMake and make; Windows builds are documented with Visual Studio or WSL. The project says it should build on most modern machines with a C++ compiler. Generated programs may loop indefinitely, so execution timeouts are recommended. The repository includes a COPYING license file. Bug reports and suggestions go through GitHub issues, with discussion on a mailing list; maintainers warn replies may be delayed.

Schemathesis generates property-based tests from OpenAPI and GraphQL schemas to help engineers find API bugs. Its cases probe boundary values, type mismatches and constraint violations, and it validates API responses against the OpenAPI specification. For multi-step workflows, it chains operations using links inferred from a schema. During a run, it can learn validation rules, resource IDs and authentication requirements from server responses, then reuse them. Users can add Python checks for business rules alongside built-in checks. It supports OpenAPI 2.0, 3.0, 3.1 and 3.2, plus GraphQL schemas. Authentication options include Bearer tokens, Basic authentication and API keys, with environment variable substitution and Python hooks for dynamic flows. Integrations include a GitHub Action, Docker image, pip or uvx installation, and pytest. Reports can be exported as JUnit, VCR, HAR, NDJSON, JSON or Allure; failures can also be saved with curl commands for replay. The tool is self-hosted and licensed under MIT.