Mutatest vs MutPy
Mutatest
mutatest.readthedocs.ioMutPy
github.comSide by side
| Figure | Mutatest | MutPy |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | Yes | — |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | Yes |
| API | Yes | — |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 1 |
| Spec score · Mutation Testing Tools | 8.7 (#1) | 7.9 (#7) |
| Integrations | The documentation describes running test commands such as pytest and using coverage data to filter mutation targets. | The documented test runner integrations are unittest and experimental pytest; no other integrations are listed. |
| Mutation method | It uses Python’s Abstract Syntax Tree grammar to produce valid mutants and changes local __pycache__ files instead of source files. | It applies mutations at the abstract syntax tree level. |
| Purpose | Mutatest is a diagnostic command-line tool for assessing test coverage by checking whether tests detect small code mutations. | MutPy is a mutation testing tool for Python source code that evaluates the quality of software tests. |
Plans
In their own words
Mutatest
Mutatest is a free command-line tool for Python projects that checks whether tests catch small changes to code. It creates valid mutations from Python’s Abstract Syntax Tree grammar and applies them to local __pycache__ files rather than changing source files. If coverage data is available, it can limit targets to lines marked as covered. Mutations are sampled randomly, with a seed available to make runs reproducible. Command-line controls include run modes, mutation categories, file exclusions, timeouts, reports, and thresholds for surviving mutants; an API supports custom controls. The documentation describes using test commands such as pytest and coverage data to filter targets. Mutatest can raise an exception when surviving mutants meet a configured threshold, including in automated runs such as CI. It supports Linux, macOS, and Windows, and is free under the MIT License. Installation requires Python 3.7 or 3.8; multiprocessing is available on 3.8, but not 3.7. Repeated test runs can be slow, so it is intended for diagnostics rather than every commit across an entire codebase.
MutPy
MutPy is a mutation testing tool for Python that checks whether a test suite catches changes to its source code. It creates mutations at the abstract syntax tree level, then runs tests to reveal which changes go undetected. The project describes compatibility with Python 3.3 and later. It uses unittest by default, while pytest support is experimental. Reports can be generated in YAML or HTML, and coverage analysis can restrict mutation to code exercised by tests. Users can also select or disable operators and sample a percentage of mutants. MutPy supports higher order mutations and includes standard and experimental operator sets. Installation is documented through pip from PyPI or from the repository source. The software is free at free and licensed under Apache License 2.0. PyPI lists version 0.6.1, released November 17, 2019, as the latest release. The README warns that disabling stdout can damage tests that interact with sys.stdout.