Cucumber.js vs Web Test Runner
Cucumber.js
cucumber.io
Web Test Runner
modern-web.devSide by side
| Figure | Cucumber.js | Web Test Runner |
|---|---|---|
| Price from | Free | Free |
| Free plan | Yes | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | Yes | Yes |
| macOS | Yes | Yes |
| Linux | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 2 | 1 |
| Spec score · JavaScript Testing Frameworks | 9.0 (#1) | 8.2 (#4) |
| Installation | The documentation shows installing `@cucumber/cucumber` as a development dependency with npm or Yarn. | The official installation instructions use npm to add @web/test-runner as a development dependency. |
| Integrations | Cucumber says it is not a browser automation tool, but works with browser automation tools such as Selenium WebDriver. | The docs describe using the Playwright launcher with GitHub Actions via Microsoft's Playwright Action. |
| Purpose | Cucumber reads executable specifications written in plain text and checks whether software conforms to them, reporting success or failure for each scenario. | Web Test Runner is a test runner for web applications. |
| Support | The FAQ directs users with questions to the Cucumber community and lists Cucumber School online training for deeper learning. | The project links to Discord for community discussion and lists sponsorship and contributions as ways to support it. |
Plans
In their own words
Cucumber.js
Cucumber.js checks software against executable specifications written as plain-text scenarios, reporting whether each scenario passes or fails. It is the JavaScript implementation of Cucumber and runs on Node.js. Scenarios use Gherkin syntax and are usually kept in `.feature` files alongside the software in source control. Step definitions connect each Gherkin step to code that carries out its described action; expressions can use Cucumber Expressions or regular expressions, but not both syntaxes in a single expression.
The tool runs feature files from the command line and can run in continuous integration, including parallel execution. Reporter plugins can produce results, and reports can be published to Cucumber Reports with `--publish`. Published reports are public and deleted after 24 hours. Cucumber.js is available as the `@cucumber/cucumber` npm module, installed as a development dependency with npm or Yarn. The setup requires Node.js and one of those package managers. It is free and open source, with Linux, macOS, Windows, and self-hosted platforms listed. It works with browser automation tools, but is not itself one.
Web Test Runner
Web Test Runner is a test runner for web applications, installable as an npm development dependency. It runs tests in parallel and isolation, giving each test file its own browser environment. Headless testing is supported through launchers including Puppeteer, Playwright, Selenium, BrowserStack, and WebdriverIO. Playwright can use Chromium, Firefox, and WebKit; BrowserStack runs tests remotely, including against older browser versions. Debug mode opens a browser window with developer tools, and the runner reports browser logs, 404 responses, and errors. Its command line offers interactive watch mode that reruns tests when files change, plus code coverage with configurable thresholds and reports. Default coverage instrumentation uses Chromium. Other features include ES module mocking through import maps and Mocha integration, with guidance for a custom test framework. The project is MIT-licensed, powered by esbuild and Rollup plugins, and directs users to GitHub and Discord for help and discussion. It is free and open source.