Cucumber.js vs Karma
Cucumber.js
cucumber.ioKarma
karma-runner.github.ioSide by side
| Figure | Cucumber.js | Karma |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| iOS | — | Yes |
| Android | — | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 2 | 1 |
| Spec score · JavaScript Testing Frameworks | 9.0 (#1) | 8.2 (#3) |
| Installation | The documentation shows installing `@cucumber/cucumber` as a development dependency with npm or Yarn. | Karma runs on Node.js, is distributed as an npm package, and the installation guide recommends installing it locally in the project. |
| Purpose | Cucumber reads executable specifications written in plain text and checks whether software conforms to them, reporting success or failure for each scenario. | Karma runs JavaScript tests in multiple real browsers and reports results to developers. |
| Support | The FAQ directs users with questions to the Cucumber community and lists Cucumber School online training for deeper learning. | The project directs questions to its mailing list or Gitter and says the issue tracker is for bug reports and feature discussions. |
Plans
In their own words
Cucumber.js
Cucumber.js checks software against executable specifications written as plain-text scenarios, reporting whether each scenario passes or fails. It is the JavaScript implementation of Cucumber and runs on Node.js. Scenarios use Gherkin syntax and are usually kept in `.feature` files alongside the software in source control. Step definitions connect each Gherkin step to code that carries out its described action; expressions can use Cucumber Expressions or regular expressions, but not both syntaxes in a single expression.
The tool runs feature files from the command line and can run in continuous integration, including parallel execution. Reporter plugins can produce results, and reports can be published to Cucumber Reports with `--publish`. Published reports are public and deleted after 24 hours. Cucumber.js is available as the `@cucumber/cucumber` npm module, installed as a development dependency with npm or Yarn. The setup requires Node.js and one of those package managers. It is free and open source, with Linux, macOS, Windows, and self-hosted platforms listed. It works with browser automation tools, but is not itself one.
Karma
Karma is a JavaScript test runner that starts a web server, connects browsers, runs tests, and reports results to developers. It can watch configured files and rerun tests when they change. Plugins connect frameworks such as Jasmine, Mocha, and QUnit, alongside preprocessors, reporters, and browser launchers. Documented launchers include Chrome, Firefox, Safari, PhantomJS, JSDOM, Opera, Internet Explorer, SauceLabs, and BrowserStack. A browser on another device on the same network, including a phone or tablet, can be captured by visiting the Karma server URL. Karma runs on Node.js and is distributed as an npm package; its installation guide recommends installing it locally in a project. Documentation includes integration guides for Travis CI, Jenkins, and Semaphore CI. The project describes Karma as primarily designed for low-level unit testing and points to Protractor for high-level testing. Karma is free and MIT-licensed, but deprecated: it accepts no new features or general bug fixes. Critical security fixes continue until 12 months after Angular CLI’s Web Test Runner support is marked stable.