Mutant vs pytest-gremlins

Mutant is Ruby mutation-testing software that changes code in controlled ways to check whether tests catch the changes. It alters arithmetic, logical and bitwise operators, removes statements, and changes return values. It works with RSpec, Minitest and Test::Unit, and supports incremental runs for CI. Supported Ruby versions are 3.2, 3.3, 3.4 and 4.0 for runtime, syntax and mutations. Rails support is verified for versions 7.2, 8.0 and 8.1, with parallel-worker database isolation for PostgreSQL and SQLite. Mutant runs on developers’ machines or in CI, and includes parallel execution and reports of surviving mutants. It cannot generate mutations for methods defined inside closures, including methods created with module_eval, class_eval, define_method or define_singleton_method. The open-source plan is free for public repositories with the full feature set. Commercial monthly pricing is $30 a month per developer; yearly pricing is $250 a year per developer. CI use requires no additional subscriptions.

pytest-gremlins is a pytest plugin for checking whether an existing test suite catches small changes to code. It runs mutation tests against Python projects and can report results in the console, HTML, or JSON, including multiple formats in one run. Getting started requires Python 3.11 or later, pytest 7.0 or later, and existing tests; installation is documented with pip, uv, and Poetry. The project’s quick-start command is `pytest --gremlins`. To manage execution, it uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and it works with pytest-xdist. Options can be set in `pyproject.toml` or with command-line flags, while inline pragmas can pardon specific mutations. A GitHub Action can run the checks and fail a workflow if its score misses a threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. It is a free, MIT-licensed package for Linux, classified as Alpha. The guide warns that an initial run may take 10–100 times as long as a normal test run, though caching can shorten later runs. It also notes that equivalent mutations can make a perfect score unattainable and unnecessary.