MuJava vs Mutatest
MuJava
albany.eduMutatest
mutatest.readthedocs.ioSide by side
| Figure | MuJava | Mutatest |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| API | — | Yes |
| Self-hosted | — | Yes |
| Plans published | 1 | 1 |
| Spec score · Mutation Testing Tools | 8.4 (#3) | 8.7 (#1) |
| Coverage | After generating mutants, µJava runs tests and evaluates their mutation coverage. | When coverage data is available, mutation targets can be restricted to lines marked as covered. |
| Mutation operators | It creates object-oriented mutants using 24 class-level operators and also supports method-level operators. | Supported operators include arithmetic, bitwise, Boolean, comparison, membership, identity, conditional and index mutations. |
| Purpose | MuJava automatically generates mutants for Java programs for traditional method-level and class-level mutation testing. | Mutatest is a diagnostic command-line tool for assessing test coverage by checking whether tests detect small code mutations. |
| Support | The maintainers offer MuJava as-is and do not guarantee question answering, fault fixing, or improvements. | The project directs users to its GitHub issue tracker to submit bugs or request features. |
Plans
In their own words
MuJava
MuJava is a free, open-source tool for mutation testing of Java programs. It generates mutants for individual classes and packages, using 24 class-level operators as well as method-level operators, then runs tests against those mutants and evaluates mutation coverage. Tests must be JUnit test cases with public test classes and methods that call the classes under test; JUnit test suites are not supported. Equivalent mutants require manual identification, and the documentation says they typically make up 5% to 20% of mutants. Version 4 supports Java 1.6 language features, including generics, annotations, enumerations, varargs, enhanced for-each loops and static imports. Setup uses two JAR files and a configuration file, with Java CLASSPATH and PATH configuration. The documented workflow uses Java GUI applications; a command-line interface called muScript was released in March 2014, and an Eclipse plugin was created in 2007. The project is provided as-is, without guarantees of answers, repairs or improvements. It is available for Linux, macOS and Windows.
Mutatest
Mutatest is a free command-line tool for Python projects that checks whether tests catch small changes to code. It creates valid mutations from Python’s Abstract Syntax Tree grammar and applies them to local __pycache__ files rather than changing source files. If coverage data is available, it can limit targets to lines marked as covered. Mutations are sampled randomly, with a seed available to make runs reproducible. Command-line controls include run modes, mutation categories, file exclusions, timeouts, reports, and thresholds for surviving mutants; an API supports custom controls. The documentation describes using test commands such as pytest and coverage data to filter targets. Mutatest can raise an exception when surviving mutants meet a configured threshold, including in automated runs such as CI. It supports Linux, macOS, and Windows, and is free under the MIT License. Installation requires Python 3.7 or 3.8; multiprocessing is available on 3.8, but not 3.7. Repeated test runs can be slow, so it is intended for diagnostics rather than every commit across an entire codebase.