FlakyWatch vs Mergify
FlakyWatch
flakywatch.devMergify
mergify.comSide by side
| Figure | FlakyWatch | Mergify |
|---|---|---|
| Price from | $29/mo | $21/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| API | — | Yes |
| Self-hosted | — | Yes |
| Plans published | 3 | 3 |
| Spec score · Test Flakiness Management Software | 7.8 (#15) | 7.9 (#12) |
| Integrations | The site lists GitHub Actions and GitHub issues; Slack integration is marked coming soon on the Team plan. | The integrations page lists GitHub, GitHub Enterprise, GitHub Actions, Jenkins, Buildkite, Slack, Datadog, Terraform, and multiple test frameworks. |
| Security | The privacy policy states that data is encrypted in transit and at rest, production systems have access controls, and the service uses least-privilege access and regular security audits. | The pricing comparison lists data encryption, SSO, and delegated roles, and identifies SOC 2 Type 2 for Max and Enterprise. |
| Support | FlakyWatch lists email support on Starter, priority support on Team, and says it aims to respond to inquiries within 48 hours during business days. | Plans list email support and Slack Connect support, while Enterprise can include optional 24/7 premium support. |
Plans
In their own words
FlakyWatch
FlakyWatch monitors GitHub Actions test results, classifies unstable tests, estimates CI and developer time costs, and tracks flakes as GitHub issues. Users upload JUnit XML through the FlakyWatch GitHub App and a workflow step; the service says agents and configuration files are unnecessary. Supported runners include pytest, Jest, JUnit, RSpec, and other JUnit-compatible tools. Tests receive stable, flaky, broken, or newly unstable labels with confidence scores. Generated GitHub issues include classification, impact, and recommended next steps, and can reopen when a flake returns. Dashboard metrics include blocked runs, main failures, affected branches, and cost per test. An optional weekly email digest reports new unstable tests, cost trends, and open issues. The free plan costs nothing and covers one repository with 14-day history. Starter costs $29 a month for five repositories and 30-day history; Team costs $79 a month for 20 repositories and 90-day history. Slack and CSV export are marked coming soon on Team.
Mergify
Mergify automates the path from pull request to a green main branch with merge queues, CI insights, flaky-test tools, and merge protections. Its queue rebases each pull request onto the latest main and merges only when CI passes against the combined state. Teams can combine pull requests into a single CI run; if that batch fails, Mergify can bisect it to locate the culprit. CI Insights reports unreliable jobs, retries them automatically, and provides data for investigating causes.
Merge Protections supports custom rules, scheduled merge windows, freeze periods, and exceptions for urgent fixes. The queue works with any CI that reports status checks to GitHub, while deeper CI Insights integrations are listed for selected providers. Integrations include GitHub, GitHub Enterprise, GitHub Actions, Jenkins, Buildkite, Slack, Datadog, and Terraform, among others. Mergify offers a free plan for open-source projects or private teams of up to five users. Max costs $21 per user per month and includes a 14-day free trial. Enterprise pricing is custom; it can include optional on-premise deployment and dedicated support. Mergify was founded in 2018 and is headquartered in Toulouse, France.