FlakyWatch vs Qualflare
FlakyWatch
flakywatch.devQualflare
qualflare.comSide by side
| Figure | FlakyWatch | Qualflare |
|---|---|---|
| Price from | $29/mo | $16/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | — | Yes |
| iOS | — | Yes |
| Android | — | Yes |
| API | — | Yes |
| Plans published | 3 | 8 |
| Spec score · Test Flakiness Management Software | 7.8 (#15) | 8.8 (#6) |
| Data retention | Test execution data is retained for 30 days by default; aggregated statistics are retained for 90 days on paid plans and 14 days on the free plan. | Project retention defaults are 30 days for Starter, 90 days for Core, one year for Scale, and custom retention for Enterprise. |
| Integrations | The site lists GitHub Actions and GitHub issues; Slack integration is marked coming soon on the Team plan. | Official integrations include GitHub, GitLab, Jenkins, Slack, Discord, Jira, and webhooks. |
| Security | The privacy policy states that data is encrypted in transit and at rest, production systems have access controls, and the service uses least-privilege access and regular security audits. | Qualflare states that it uses TLS 1.2+ in transit, AES-256 at rest, role-based access control, mandatory SSO and MFA for staff, and immutable employee-access audit logs. |
| Support | FlakyWatch lists email support on Starter, priority support on Team, and says it aims to respond to inquiries within 48 hours during business days. | Starter includes community support, Core and Scale include priority email support, and Enterprise includes 24/7 customer support and a 99.9% SLA. |
| What it does | FlakyWatch monitors GitHub Actions test results, detects and classifies flaky tests, estimates CI and developer time costs, and tracks them as GitHub issues. | Qualflare is a hosted AI test-management and observability service that collects CI results, groups failures by likely root cause, detects flaky tests, and estimates release risk. |
Plans
Qualflare
- Open source planFree
- StarterFree
- StarterFree
- Core$16 a month
- Core$16 a month
- Scale$48 a month
- Scale$48 a month
- EnterprisePriced on request
In their own words
FlakyWatch
FlakyWatch monitors GitHub Actions test results, classifies unstable tests, estimates CI and developer time costs, and tracks flakes as GitHub issues. Users upload JUnit XML through the FlakyWatch GitHub App and a workflow step; the service says agents and configuration files are unnecessary. Supported runners include pytest, Jest, JUnit, RSpec, and other JUnit-compatible tools. Tests receive stable, flaky, broken, or newly unstable labels with confidence scores. Generated GitHub issues include classification, impact, and recommended next steps, and can reopen when a flake returns. Dashboard metrics include blocked runs, main failures, affected branches, and cost per test. An optional weekly email digest reports new unstable tests, cost trends, and open issues. The free plan costs nothing and covers one repository with 14-day history. Starter costs $29 a month for five repositories and 30-day history; Team costs $79 a month for 20 repositories and 90-day history. Slack and CSV export are marked coming soon on Team.
Qualflare
Qualflare is an AI-powered test management and observability platform that collects CI results, clusters failures, detects flaky tests and scores release risk. Teams can manage manual tests in its web application and upload automated results through its CLI, which parses results from 23 testing frameworks. Supported tools include JUnit, pytest, Jest, Playwright, Cypress, Newman, k6 and security tools. Its Quo assistant analyzes coverage gaps, suggests test cases, generates test steps and answers questions about test-management data. AI Failure Clusters group similar failures and offer root-cause summaries, fix suggestions and severity information. Integrations include GitHub, GitLab, Jira, Slack, Discord, Jenkins and custom webhooks. A REST API provides access to test data using project-scoped tokens. Qualflare says customer data stays in the European Union and describes its encryption and access controls, while noting that it does not currently hold SOC 2 Type II or ISO 27001 certification. Mobile test results can be ingested, but the service does not provision devices or run mobile tests. Plans include a free option; paid pricing starts at $16/user/mo (annual), and a 14-day trial is listed.