mutmut vs pytest-gremlins

mutmut is a free, open-source mutation testing system for Python. It makes small code changes called mutants so developers can check whether tests detect them. Its listed operators increment integer literals by one, change < to <=, and swap break and continue. The project supports pytest and remembers completed work for incremental runs, selecting tests to run for each mutant. An interactive terminal interface lets users browse mutants, write one to disk, and rerun individual cases after adding tests. Runs are described as parallel; configured coverage.py can restrict mutations to covered lines. The documentation also supports filtering with mypy or pyrefly, but not pyright or ty. For CI/CD, mutmut can export statistics and generate mutation-score badge JSON for a Shields endpoint. Configuration in setup.cfg or pyproject.toml can specify source paths and pytest selection arguments. It requires Python 3.10 or later and a system with fork support, so Windows users need WSL. Version 3 and later mutate code inside functions; version 2 uses a different model for code outside functions. The package is licensed under BSD-3-Clause.

pytest-gremlins is a pytest plugin for checking whether an existing test suite catches small changes to code. It runs mutation tests against Python projects and can report results in the console, HTML, or JSON, including multiple formats in one run. Getting started requires Python 3.11 or later, pytest 7.0 or later, and existing tests; installation is documented with pip, uv, and Poetry. The project’s quick-start command is `pytest --gremlins`. To manage execution, it uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and it works with pytest-xdist. Options can be set in `pyproject.toml` or with command-line flags, while inline pragmas can pardon specific mutations. A GitHub Action can run the checks and fail a workflow if its score misses a threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. It is a free, MIT-licensed package for Linux, classified as Alpha. The guide warns that an initial run may take 10–100 times as long as a normal test run, though caching can shorten later runs. It also notes that equivalent mutations can make a perfect score unattainable and unnecessary.