Skip to content
SPECIFICTION
Get listed

Mutatest

mutatest.readthedocs.io

Free plan Open source WindowsmacOSLinuxAPISelf-hosted
Visit mutatest.readthedocs.io
Spec score 8.7 #1 of 25 in Mutation Testing Tools

What it is

Mutatest is a free command-line tool for Python projects that checks whether tests catch small changes to code. It creates valid mutations from Python’s Abstract Syntax Tree grammar and applies them to local __pycache__ files rather than changing source files. If coverage data is available, it can limit targets to lines marked as covered. Mutations are sampled randomly, with a seed available to make runs reproducible. Command-line controls include run modes, mutation categories, file exclusions, timeouts, reports, and thresholds for surviving mutants; an API supports custom controls. The documentation describes using test commands such as pytest and coverage data to filter targets. Mutatest can raise an exception when surviving mutants meet a configured threshold, including in automated runs such as CI. It supports Linux, macOS, and Windows, and is free under the MIT License. Installation requires Python 3.7 or 3.8; multiprocessing is available on 3.8, but not 3.7. Repeated test runs can be slow, so it is intended for diagnostics rather than every commit across an entire codebase.

Spec sheet

What it does
Mutatest evaluates Python test quality by introducing small code mutations and checking whether the test suite detects them.Read 15 Oct 2026 · mutatest.readthedocs.io
Purpose
Mutatest is a diagnostic command-line tool for assessing test coverage by checking whether tests detect small code mutations.Read 2 Oct 2026 · mutatest.readthedocs.io
API
Mutatest includes an API for custom mutation controls.Read 2 Oct 2026 · mutatest.readthedocs.io
Integrations
The documentation describes running test commands such as pytest and using coverage data to filter mutation targets.Read 2 Oct 2026 · mutatest.readthedocs.io
Support
The project directs users to its GitHub issue tracker to submit bugs or request features.Read 2 Oct 2026 · github.com
API objects
The documented API centers on Genome, GenomeGroup and Mutant objects.Read 15 Oct 2026 · mutatest.readthedocs.io
AST mutation engine
Mutations are generated from Python's Abstract Syntax Tree grammar.Read 15 Oct 2026 · mutatest.readthedocs.io
CI use
Mutatest can raise an exception when surviving mutants meet a configured threshold, for use in automated runs such as a continuous integration stage.Read 2 Oct 2026 · mutatest.readthedocs.io
CLI configuration
Settings can be supplied through mutatest.ini or setup.cfg sections named mutatest or tool:mutatest.Read 15 Oct 2026 · github.com
Controls
The command line supports run modes, mutation category selection, file exclusions, timeouts, output reports, and survivor thresholds.Read 2 Oct 2026 · mutatest.readthedocs.io
Coverage
When coverage data is available, mutation targets can be restricted to lines marked as covered.Read 2 Oct 2026 · mutatest.readthedocs.io
Coverage guidance
Coverage data can restrict mutation targets to code executed by tests.Read 15 Oct 2026 · mutatest.readthedocs.io
Custom API
The package exposes an API for custom mutation controls.Read 15 Oct 2026 · mutatest.readthedocs.io
Diagnostic positioning
The installation guide describes Mutatest as a diagnostic tool and cautions that running a full suite repeatedly can be slow for continuous integration.Read 15 Oct 2026 · github.com
Documentation depth
Documentation includes installation, command-line controls, API reference, API tutorial, mutation operators and contributing guidance.Read 15 Oct 2026 · mutatest.readthedocs.io
Execution modes
The command line supports fast and detection or survival stopping modes, including f, s, d and sd.Read 15 Oct 2026 · github.com
Independent activity record
Ecosyste.ms reported the last repository push as February 17, 2023.Read 15 Oct 2026 · awesome.ecosyste.ms
Independent popularity record
Ecosyste.ms reported 97 stars, 18 forks and two watchers when it last synced the repository.Read 15 Oct 2026 · awesome.ecosyste.ms
Independent repository record
Ecosyste.ms records the repository as created on December 22, 2018, with MIT licensing.Read 15 Oct 2026 · awesome.ecosyste.ms
Install channels
Installation is documented through pip, conda-forge or a local source checkout.Read 15 Oct 2026 · mutatest.readthedocs.io
Known limitation
The documentation warns that repeated test-suite runs can make mutation testing slow, so it is intended as a diagnostic tool rather than a run over an entire codebase on every commit.Read 2 Oct 2026 · mutatest.readthedocs.io
Latest release
PyPI lists version 3.1.0 as released on February 20, 2022.Read 15 Oct 2026 · pypi.org
Licence
PyPI identifies the project as MIT licensed and OSI-approved.Read 15 Oct 2026 · pypi.org
Limit: diagnostics
The public issue tracker contains an open report about being unable to retrieve failing-test information for mutants, opened May 28, 2025.Read 15 Oct 2026 · github.com
Limit: maintenance
The public issue tracker contains an open request to support newer Python versions, opened November 30, 2025.Read 15 Oct 2026 · github.com
Limit: mutation coverage
An open issue reports that type hints are not ignored as mutation targets.Read 15 Oct 2026 · github.com
Limit: parallel integration
An independent Reddit comparison says Mutatest lacks pytest-xdist integration and that its last PyPI release was in 2022.Read 15 Oct 2026 · reddit.com
Maintainer
PyPI names Evan Kepner as the author and lists one maintainer account, ekepner.Read 15 Oct 2026 · pypi.org
Mutation method
It uses Python’s Abstract Syntax Tree grammar to produce valid mutants and changes local __pycache__ files instead of source files.Read 2 Oct 2026 · mutatest.readthedocs.io
Mutation operators
Supported operators include arithmetic, bitwise, Boolean, comparison, membership, identity, conditional and index mutations.Read 15 Oct 2026 · github.com
Notable release history
The repository release page records version 3.0.0 as a stable release and earlier releases adding setup.cfg configuration and parallel-cache fixes.Read 15 Oct 2026 · github.com
Operating systems
The project documentation says it was tested on Linux, Windows and macOS with Azure pipelines.Read 15 Oct 2026 · mutatest.readthedocs.io
Package or file scope
Mutation runs can target an entire package or one source file.Read 15 Oct 2026 · mutatest.readthedocs.io
Parallel execution
Multiprocessing is supported on Python 3.8; parallelization is not supported on Python 3.7.Read 2 Oct 2026 · mutatest.readthedocs.io
Parallel trials
Multiprocessing parallel trials are supported on Python 3.8 through the parallel option.Read 15 Oct 2026 · github.com
Platform support
The project says it was tested on Linux, Windows, and MacOS with Azure pipelines.Read 2 Oct 2026 · mutatest.readthedocs.io
Praise: source safety
An independent Reddit comparison highlights Mutatest's AST approach and its avoidance of source-file changes.Read 15 Oct 2026 · reddit.com
Pytest integration
Pytest is auto-detected for conventional tests and is the only test runner documented as tested.Read 15 Oct 2026 · github.com
Python package metadata
PyPI lists Python 3.7 or newer as the requirement, with Python 3.7 and 3.8 classifiers.Read 15 Oct 2026 · pypi.org
Python support
The installation documentation requires Python 3.7 or Python 3.8.Read 2 Oct 2026 · mutatest.readthedocs.io
Random sampling
Users can choose mutation-location sample size and a random seed for repeatable trials.Read 15 Oct 2026 · github.com
Reporting
The API reference documents result summaries containing mutation results, identified and mutated-location counts and total runtime.Read 15 Oct 2026 · mutatest.readthedocs.io
Sampling
It randomly samples mutation locations and potential mutations, and a random seed can make trials reproducible.Read 2 Oct 2026 · mutatest.readthedocs.io
Source protection
Mutation trials write changes to local __pycache__ files instead of modifying source files.Read 15 Oct 2026 · mutatest.readthedocs.io
System requirement
The documented runtime requirement is Python 3.7 or 3.8.Read 15 Oct 2026 · mutatest.readthedocs.io
Test runner limitation
Although a test command can be supplied, the documentation says only pytest has been tested to date.Read 2 Oct 2026 · mutatest.readthedocs.io

Plans and prices

Mutatest Free Free and open source under the MIT License Read 2 Oct 2026 · mutatest.readthedocs.io

Every Mutatest plan and price →

Strong points and limits

Strong points

  • Can target lines marked covered when coverage data exists.
  • A random seed can make trials reproducible.
  • Configurable survivor thresholds support automated runs.
  • Includes an API for custom mutation controls.

Limits

  • Repeated test-suite runs can be slow.
  • Only pytest has been tested to date.
  • Python 3.7 does not support parallelization.

Who it suits

Mutatest suits Python developers who want to check whether their tests detect small code changes, including in automated runs. It is a diagnostic tool rather than a routine whole-codebase check on every commit.

Verdict

Mutatest offers configurable mutation checks and a threshold for flagging surviving mutants. Its documented test-runner coverage is limited to pytest, and repeated runs may take time.

Where Specifiction places it

Alternatives to MutatestAll alternatives →