pytest-gremlins
github.com
What it is
pytest-gremlins is a pytest plugin for checking whether an existing test suite catches small changes to code. It runs mutation tests against Python projects and can report results in the console, HTML, or JSON, including multiple formats in one run. Getting started requires Python 3.11 or later, pytest 7.0 or later, and existing tests; installation is documented with pip, uv, and Poetry. The project’s quick-start command is `pytest --gremlins`. To manage execution, it uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution, and it works with pytest-xdist. Options can be set in `pyproject.toml` or with command-line flags, while inline pragmas can pardon specific mutations. A GitHub Action can run the checks and fail a workflow if its score misses a threshold; CI examples also cover GitLab CI, CircleCI, and Dagger. It is a free, MIT-licensed package for Linux, classified as Alpha. The guide warns that an initial run may take 10–100 times as long as a normal test run, though caching can shorten later runs. It also notes that equivalent mutations can make a perfect score unattainable and unnecessary.
Spec sheet
- Purpose
- pytest-gremlins is a pytest plugin that injects small code mutations and checks whether tests catch them.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Audience
- The package metadata lists developers as its intended audience.Read 7 Oct 2026 · pypi.org
- Integrations
- The documentation describes compatibility with pytest-xdist and a GitHub Action for running mutation tests and enforcing a score threshold.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- License
- The project is MIT licensed.Read 7 Oct 2026 · github.com
- CI integrations
- The GitHub Action documentation points to manual CI workflow examples for GitHub Actions, GitLab CI, CircleCI, and Dagger.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- CI options
- The CI integration guide covers GitHub Actions, GitLab CI, CircleCI, and Dagger.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Configuration
- Options can be configured through `pyproject.toml` or command-line flags, and mutations can be pardoned with an inline pragma.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Development status
- The package metadata classifies its development status as Alpha.Read 7 Oct 2026 · pypi.org
- GitHub integration
- The composite GitHub Action installs a project, runs mutation testing, and can fail a workflow when the mutation score is below a threshold.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Installation
- The project documents installation with pip, uv, and Poetry.Read 7 Oct 2026 · github.com
- Installation requirements
- The getting started guide requires Python 3.11 or later, pytest 7.0 or later, and a project with existing tests.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Limits
- The guide says equivalent mutations can make a 100% mutation score impossible and not worth pursuing.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Mutation control
- Users can suppress equivalent or untestable mutations inline with `# gremlin: pardon[reason]`.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Publishing provenance
- PyPI reports that the October 6, 2026 release was uploaded using Trusted Publishing and has a verified attestation signed by GitHub Actions.Read 7 Oct 2026 · pypi.org
- pytest integration
- The plugin works with existing pytest test suites and pytest-xdist, including `-n auto`.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Python requirement
- The package requires Python 3.11 or later.Read 7 Oct 2026 · pypi.org
- Reports
- It supports console, HTML, and JSON reports, including multiple report formats in one run.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Runtime
- The guide says a first mutation-testing run may take 10–100 times as long as a normal test run; caching can make later runs faster.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Security and license
- The repository identifies the project as MIT-licensed; the license text provides the software “AS IS” without warranty.Read 7 Oct 2026 · github.com
- Setup
- The project says mutation testing can be started with `pip install pytest-gremlins` and `pytest --gremlins`.Read 7 Oct 2026 · github.com
- Speed
- Its speed features include mutation switching, coverage-guided test selection, incremental analysis, and parallel execution.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
- Speed features
- It uses mutation switching, coverage-guided test selection, incremental analysis, and parallel execution.Read 7 Oct 2026 · pytest-gremlins.readthedocs.io
Plans and prices
pytest-gremlins Free MIT-licensed Python package · requires Python 3.11+ Read 7 Oct 2026 · github.com
Strong points and limits
Strong points
- Works with pytest-xdist, including parallel execution.
- Offers console, HTML, and JSON reports.
- Configurable through pyproject.toml or command-line flags.
- GitHub Action can enforce a mutation-score threshold.
Limits
- Requires Python 3.11 or later and pytest 7.0 or later.
- A first run may take 10–100 times as long as normal tests.
- Equivalent mutations can prevent a 100% score.
Who it suits
It suits Python teams using pytest that want to assess tests around business logic, security-critical code, or financial calculations. Teams should be prepared for a potentially much longer first run.
Verdict
pytest-gremlins adds mutation testing to existing pytest projects, with several ways to manage runs and inspect results. Account for its Python and pytest requirements and the possibility of a much slower first run.
Where Specifiction places it
- Mutation Testing Tools#10 of 257.7
