MutPy vs PIT

MutPy is a mutation testing tool for Python that checks whether a test suite catches changes to its source code. It creates mutations at the abstract syntax tree level, then runs tests to reveal which changes go undetected. The project describes compatibility with Python 3.3 and later. It uses unittest by default, while pytest support is experimental. Reports can be generated in YAML or HTML, and coverage analysis can restrict mutation to code exercised by tests. Users can also select or disable operators and sample a percentage of mutants. MutPy supports higher order mutations and includes standard and experimental operator sets. Installation is documented through pip from PyPI or from the repository source. The software is free at free and licensed under Apache License 2.0. PyPI lists version 0.6.1, released November 17, 2019, as the latest release. The README warns that disabling stdout can damage tests that interact with sys.stdout.

PIT is a free mutation-testing tool for Java and JVM bytecode projects. It measures whether unit tests detect faults by running them against modified versions of application code. Configurable bytecode mutations can remove method calls, invert logic, or change return values. PIT selects and prioritizes tests using line coverage, execution speed, and naming conventions, then reports mutation coverage alongside line coverage. It can run from the command line, Ant, or Maven; third-party integrations are listed for Gradle, Eclipse, and IntelliJ. Java is supported directly, and Kotlin use is available through the ArcMutate plugin. PIT supports JUnit 3, JUnit 4, and TestNG; JUnit 5 needs a plugin. Versions 1.4.0 and later require Java 8 or above and JUnit or TestNG on the classpath. Mutation runs can take a long time depending on codebase size and test-suite speed, so PIT recommends focusing on changed code. It mutates bytecode and requires line numbers and source file names in that bytecode.