cargo-mutants vs pytest-gremlins
cargo-mutants
mutants.rspytest-gremlins
github.comSide by side
| Figure | cargo-mutants | pytest-gremlins |
|---|---|---|
| Price from | — | Free |
| Free plan | — | Yes |
| Free trial | — | — |
| Open source | — | — |
| Linux | Yes | Yes |
| Plans published | 0 | 1 |
| Spec score · Mutation Testing Tools | 7.5 (#12) | 7.7 (#10) |
| Configuration | Options can be configured in `.cargo/mutants.toml`, and the guide recommends checking that file into the source tree. | Options can be configured through `pyproject.toml` or command-line flags, and mutations can be pardoned with an inline pragma. |
| Installation | The guide documents installation with `cargo install --locked cargo-mutants`, `cargo binstall`, or binaries from GitHub releases. | The project documents installation with pip, uv, and Poetry. |
| Purpose | cargo-mutants is a mutation testing tool for Rust that finds places where bugs could be inserted without causing tests to fail. | pytest-gremlins is a pytest plugin that injects small code mutations and checks whether tests catch them. |
| Runtime | A run’s duration depends roughly on the number of viable mutations multiplied by the time to run the test suite for each mutation. | The guide says a first mutation-testing run may take 10–100 times as long as a normal test run; caching can make later runs faster. |
| Setup | The tool requires no changes to the source tree or other setup before installation and use. | The project says mutation testing can be started with `pip install pytest-gremlins` and `pytest --gremlins`. |
Plans
In their own words
cargo-mutants
cargo-mutants has no write-up on Specifiction yet; its figures are above.
pytest-gremlins
pytest-gremlins is a pytest plugin for checking whether an existing test suite catches small changes to code. It runs mutation tests against Python projects and can report results in the console, HTML, or JSON, including multiple formats in one run. Getting started requires Python 3.11 or later, pytest 7.0 or later, and existing tests; installation is documented with pip, uv, and Poetry. The project’s quick-start command is `pytest --gremlins`. To manage execution, it uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and it works with pytest-xdist. Options can be set in `pyproject.toml` or with command-line flags, while inline pragmas can pardon specific mutations. A GitHub Action can run the checks and fail a workflow if its score misses a threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. It is a free, MIT-licensed package for Linux, classified as Alpha. The guide warns that an initial run may take 10–100 times as long as a normal test run, though caching can shorten later runs. It also notes that equivalent mutations can make a perfect score unattainable and unnecessary.