Cosmic Ray vs Mutatest
Cosmic Ray
cosmic-ray.readthedocs.ioMutatest
mutatest.readthedocs.ioSide by side
| Figure | Cosmic Ray | Mutatest |
|---|---|---|
| Price from | — | Free |
| Free plan | — | Yes |
| Free trial | — | — |
| Open source | — | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | Yes | Yes |
| API | — | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 0 | 1 |
| Spec score · Mutation Testing Tools | 7.5 (#13) | 8.7 (#1) |
| Execution modes | The local distributor runs tests serially on the local machine, and the HTTP distributor sends work to local or remote workers that can run tests in parallel. | The command line supports fast and detection or survival stopping modes, including f, s, d and sd. |
| Parallel execution | The HTTP distributor sends mutation and test requests to workers that can run locally or remotely; each worker needs its own copy of the code under test. | Multiprocessing is supported on Python 3.8; parallelization is not supported on Python 3.7. |
| Purpose | Cosmic Ray is a mutation testing tool for Python 3 that makes small changes to production code and runs the test suite against each change. | Mutatest is a diagnostic command-line tool for assessing test coverage by checking whether tests detect small code mutations. |
Plans
In their own words
Cosmic Ray
Cosmic Ray has no write-up on Specifiction yet; its figures are above.
Mutatest
Mutatest is a free command-line tool for Python projects that checks whether tests catch small changes to code. It creates valid mutations from Python’s Abstract Syntax Tree grammar and applies them to local __pycache__ files rather than changing source files. If coverage data is available, it can limit targets to lines marked as covered. Mutations are sampled randomly, with a seed available to make runs reproducible. Command-line controls include run modes, mutation categories, file exclusions, timeouts, reports, and thresholds for surviving mutants; an API supports custom controls. The documentation describes using test commands such as pytest and coverage data to filter targets. Mutatest can raise an exception when surviving mutants meet a configured threshold, including in automated runs such as CI. It supports Linux, macOS, and Windows, and is free under the MIT License. Installation requires Python 3.7 or 3.8; multiprocessing is available on 3.8, but not 3.7. Repeated test runs can be slow, so it is intended for diagnostics rather than every commit across an entire codebase.