Captain vs Flaky Test Detective
Captain
rwx.comFlaky Test Detective
github.comSide by side
| Figure | Captain | Flaky Test Detective |
|---|---|---|
| Price from | Free | — |
| Free plan | Yes | — |
| Free trial | — | — |
| Open source | Yes | — |
| Web | Yes | — |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | — |
| API | Yes | — |
| Plans published | 4 | 0 |
| Spec score · Test Flakiness Management Software | 9.4 (#3) | 6.5 (#29) |
| License | The public Captain repository is released under the MIT license. | The repository identifies its license as MIT. |
| Quarantine | Quarantined test failures can be prevented from failing the build, allowing teams to keep shipping while fixing the tests. | Quarantine mode tags tests above a threshold and outputs a pattern usable with Playwright's --grep-invert option. |
Plans
In their own words
Captain
Captain is an open-source command-line tool for finding flaky tests and managing test execution. It measures test reliability, can retry failed tests, and can quarantine flaky ones to unblock builds. By tracking test runtimes, it balances files across parallel jobs. Framework integrations include xUnit, ExUnit, Ginkgo, go test, Bun, Cucumber, Cypress, Jest, Karma, Mocha, Playwright, TestCafe, Vitest, PHPUnit, pytest, unittest, minitest, and RSpec. CI integrations listed include RWX, GitHub Actions, CircleCI, Buildkite, and GitLab. Framework results can be reviewed in a web interface for managing flakes, retries, quarantine, and suite partitioning. Feature availability varies: xUnit does not include partitioning, and Karma does not list retries or partitioning. The free OSS plan includes file-based configuration, partitioning, flaky-test tracking and quarantine, and retries. Pro costs what you agree with the maker, billed per million test results uploaded, and adds analytics on flaky tests and performance trends with 90-day retention.
Flaky Test Detective
Flaky Test Detective has no write-up on Specifiction yet; its figures are above.