Gambit vs PIT

Gambit is a free, open-source mutation testing tool for Solidity projects. It generates altered contract versions, called mutants, to measure whether a test suite or formal verification specification detects faults. Gambit locates mutation points by traversing a program’s abstract syntax tree and applying predefined syntax transformations. A declarative configuration language lets users tailor mutations to specific parts of a program. Results can be viewed in a dashboard that summarizes mutant verification and calculates a specification score. Gambit can work with the Certora Prover to evaluate formal specifications; that workflow generates mutants and submits a verification job for each to Certora’s server. The command line includes `mutate` for generating mutants and `summary` for presenting them. A compatible `solc` binary is required for the project being mutated. Gambit supports Solidity and is listed for Linux, macOS, and Windows, though prebuilt binaries are available for Linux x86-64 and Mac; Windows and Linux ARM users must build from source. Function-call and swap-arguments-function mutation operators are disabled. The repository identifies the license as MIT. Certora directs product questions to its Help Desk channel on Discord.

PIT is a free mutation-testing tool for Java and JVM bytecode projects. It measures whether unit tests detect faults by running them against modified versions of application code. Configurable bytecode mutations can remove method calls, invert logic, or change return values. PIT selects and prioritizes tests using line coverage, execution speed, and naming conventions, then reports mutation coverage alongside line coverage. It can run from the command line, Ant, or Maven; third-party integrations are listed for Gradle, Eclipse, and IntelliJ. Java is supported directly, and Kotlin use is available through the ArcMutate plugin. PIT supports JUnit 3, JUnit 4, and TestNG; JUnit 5 needs a plugin. Versions 1.4.0 and later require Java 8 or above and JUnit or TestNG on the classpath. Mutation runs can take a long time depending on codebase size and test-suite speed, so PIT recommends focusing on changed code. It mutates bytecode and requires line numbers and source file names in that bytecode.