Karma vs Web Test Runner
Karma
karma-runner.github.io
Web Test Runner
modern-web.devSide by side
| Figure | Karma | Web Test Runner |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| iOS | Yes | — |
| Android | Yes | — |
| Self-hosted | Yes | Yes |
| Plans published | 1 | 1 |
| Spec score · JavaScript Testing Frameworks | 8.2 (#3) | 8.2 (#4) |
| Browser launchers | Documented launchers include Chrome, Firefox, Safari, PhantomJS, JSDOM, Opera, Internet Explorer, SauceLabs, and BrowserStack. | Documented launchers include Puppeteer, Playwright, Selenium, BrowserStack, and WebdriverIO. |
| Installation | Karma runs on Node.js, is distributed as an npm package, and the installation guide recommends installing it locally in the project. | The official installation instructions use npm to add @web/test-runner as a development dependency. |
| License | Karma is released under the MIT license. | The project's GitHub page identifies the repository license as MIT. |
| Purpose | Karma runs JavaScript tests in multiple real browsers and reports results to developers. | Web Test Runner is a test runner for web applications. |
| Support | The project directs questions to its mailing list or Gitter and says the issue tracker is for bug reports and feature discussions. | The project links to Discord for community discussion and lists sponsorship and contributions as ways to support it. |
Plans
In their own words
Karma
Karma is a JavaScript test runner that starts a web server, connects browsers, runs tests, and reports results to developers. It can watch configured files and rerun tests when they change. Plugins connect frameworks such as Jasmine, Mocha, and QUnit, alongside preprocessors, reporters, and browser launchers. Documented launchers include Chrome, Firefox, Safari, PhantomJS, JSDOM, Opera, Internet Explorer, SauceLabs, and BrowserStack. A browser on another device on the same network, including a phone or tablet, can be captured by visiting the Karma server URL. Karma runs on Node.js and is distributed as an npm package; its installation guide recommends installing it locally in a project. Documentation includes integration guides for Travis CI, Jenkins, and Semaphore CI. The project describes Karma as primarily designed for low-level unit testing and points to Protractor for high-level testing. Karma is free and MIT-licensed, but deprecated: it accepts no new features or general bug fixes. Critical security fixes continue until 12 months after Angular CLI’s Web Test Runner support is marked stable.
Web Test Runner
Web Test Runner is a test runner for web applications, installable as an npm development dependency. It runs tests in parallel and isolation, giving each test file its own browser environment. Headless testing is supported through launchers including Puppeteer, Playwright, Selenium, BrowserStack, and WebdriverIO. Playwright can use Chromium, Firefox, and WebKit; BrowserStack runs tests remotely, including against older browser versions. Debug mode opens a browser window with developer tools, and the runner reports browser logs, 404 responses, and errors. Its command line offers interactive watch mode that reruns tests when files change, plus code coverage with configurable thresholds and reports. Default coverage instrumentation uses Chromium. Other features include ES module mocking through import maps and Mocha integration, with guidance for a custom test framework. The project is MIT-licensed, powered by esbuild and Rollup plugins, and directs users to GitHub and Discord for help and discussion. It is free and open source.