Coverage.py vs LLVM llvm-cov
Coverage.py
coverage.readthedocs.ioLLVM llvm-cov
llvm.orgSide by side
| Figure | Coverage.py | LLVM llvm-cov |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | Yes | — |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| API | Yes | — |
| Self-hosted | Yes | — |
| Plans published | 2 | 1 |
| Spec score · Test Coverage Tools | 8.9 (#2) | 7.5 (#17) |
| Integrations | Official examples cover pytest, unittest, and nose, and pytest commonly uses the pytest-cov plugin. | The documentation describes using llvm-profdata to merge raw or indexed profiles and exporting coverage in lcov format. |
| Purpose | Coverage.py measures which parts of Python programs ran and identifies executable code that did not run, commonly to gauge test effectiveness. | llvm-cov shows code coverage information for programs instrumented to emit profile data. |
| Security | The project directs vulnerability reports to Tidelift, which it says will coordinate the fix and disclosure. | LLVM says its language frontends, analyzers, optimizers, and code generators are not hardened against malicious input, which can cause arbitrary code execution in LLVM. |
Plans
In their own words
Coverage.py
Coverage.py measures which parts of Python programs run and highlights executable code that remains untouched, helping developers assess what their tests exercise. It measures statement coverage by default and can also track branch coverage. Reports are available as text, HTML, XML, LCOV, and JSON. Dynamic contexts can associate lines of execution with the test function that ran them. Coverage.py can run test suites through their usual command; pytest-cov is available but the documentation says most users do not need it. Third-party plugins can extend measurement to non-Python files, such as connecting executed Python to Django templates. Data is stored in a SQLite database, and advanced use can rely on its API. The current documentation lists Python 3.10 through 3.15 rc2 and PyPy3 versions 3.10 and 3.11. The tool provides ctrace, pytrace, and sysmon measurement cores; sysmon is faster for simple line and branch coverage but lacks support for dynamic contexts and plugins. Coverage.py is licensed under Apache 2.0 and runs on Linux, macOS, and Windows, with self-hosting available. It is free.
LLVM llvm-cov
LLVM llvm-cov is a free, self-hosted developer tool for examining coverage in instrumented C, C++, Objective-C, and Objective-C++ programs. It accepts executables, object files, dynamic libraries, or archives and works with gcov-style data or Clang instrumentation-based profiling. Its commands include gcov, show, report, and export. Show provides line-level coverage and can focus on selected source files. Report summarizes measurements such as branch, function, and MC/DC statistics; documented coverage types also include instantiation and region coverage. Output can be text, HTML, JSON, or LCOV. JSON exports may contain regions, functions, branches, expansions, and summaries. A typical workflow builds an instrumented program, runs it to create profile data, then uses llvm-profdata merge before analysis. The gcov command is compatible with gcov from GCC 4.2 and may work with later versions. LLVM documents llvm-cov as part of its developer toolchain. It runs on Linux, macOS, and Windows and has no paid plans or stated usage limits.