Flakiness.io vs Trunk Flaky Tests
Flakiness.io
flakiness.io
Trunk Flaky Tests
trunk.ioSide by side
| Figure | Flakiness.io | Trunk Flaky Tests |
|---|---|---|
| Price from | $49/mo | Free |
| Free plan | — | Yes |
| Free trial | — | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 4 | 3 |
| Spec score · CI Failure Analysis And Flaky Test Tools | 7.4 (#8) | 7.2 (#11) |
| Spec score · Test Flakiness Management Software | 7.6 (#18) | 7.2 (#23) |
| Spec score · Test Analytics Software | 7.6 (#18) | 7.2 (#24) |
| Purpose | Flakiness.io provides test analytics for GitHub and GitLab and tracks test performance across commits to detect regressions. | Trunk detects, quarantines, and helps eliminate flaky tests across languages, test runners, and CI providers. |
| Security | The privacy policy lists encryption in transit, Cloudflare network protection, access controls, environment isolation, and monitoring; customers are responsible for securing self-hosted deployments and credentials. | Trunk says it maintains SOC 2 Type I and Type II compliance and encrypts customer data at rest with AES-256 and data in transit with TLS and HSTS. |
| Support | The pricing page lists standard support for Free and Startup plans and priority support for Growth and Enterprise plans. | The pricing page lists onboarding support for Team and Slack Connect support and dedicated support for Enterprise. |
Plans
Flakiness.io
- Startup$49 a month
- Growth$199 a month
- EnterprisePriced on request
- FreePrice not published
Trunk Flaky Tests
- FreeFree
- EnterprisePriced on request
- TeamPrice not published
In their own words
Flakiness.io
Flakiness.io provides test analytics for GitHub and GitLab, tracking outcomes across commits to distinguish flaky tests from regressions. It labels a test that alternates between passing and failing on the same commit as flaky, while persistent outcome changes at a commit count as regressions. Listed runners include Playwright, Pytest, Vitest, Jest, CucumberJS, Bun, Rust, and a JUnit XML bridge; a Node.js SDK supports custom integrations. The service works with any CI provider, ingests results as they arrive, merges shards, and separates staging and production histories. Reports include test history, grouped errors, system telemetry, and a test waterfall. Artifacts can include logs, screenshots, videos, traces, and HTML files. Recorded durations can help Playwright balance shards using historical runtime. Hosted plans include a Free option with 1 GB of artifact storage and 90 days of retention; Startup costs $49 a month with 10 GB and 180 days. Growth costs $199 a month. Self-hosting uses PostgreSQL and an S3-compatible object store. Customers must not upload specified sensitive data.
Trunk Flaky Tests
Trunk Flaky Tests has no write-up on Specifiction yet; its figures are above.