muttest vs pytest-gremlins

muttest is a free R package for mutation testing: it changes selected parts of R code and runs tests to measure which changes the test suite catches. Its mutation score reports the percentage of mutations caught, helping identify weak assertions. Users can choose what to change and where, with mutators for operators, boolean and NA or NULL values, function calls, strings, numbers, indexes, conditions, negation, and return values. Presets group common mutation types, including arithmetic, comparison, logical, numeric, string, and condition changes. Examples use testthat and show how changing comparison operators can expose untested boundary cases. Reports are available in HTML and JSON, with feedback on files that may need stronger assertions. Documentation links a guide for running checks in CI, including GitHub Actions, score badges, and thresholds. A July 2026 update describes parallel execution. The package supports Linux, macOS, and Windows, is licensed under MIT plus a LICENSE file, and requires R version 4.1.0 or later according to its CRAN listing.

pytest-gremlins is a pytest plugin for checking whether an existing test suite catches small changes to code. It runs mutation tests against Python projects and can report results in the console, HTML, or JSON, including multiple formats in one run. Getting started requires Python 3.11 or later, pytest 7.0 or later, and existing tests; installation is documented with pip, uv, and Poetry. The project’s quick-start command is `pytest --gremlins`. To manage execution, it uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and it works with pytest-xdist. Options can be set in `pyproject.toml` or with command-line flags, while inline pragmas can pardon specific mutations. A GitHub Action can run the checks and fail a workflow if its score misses a threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. It is a free, MIT-licensed package for Linux, classified as Alpha. The guide warns that an initial run may take 10–100 times as long as a normal test run, though caching can shorten later runs. It also notes that equivalent mutations can make a perfect score unattainable and unnecessary.