FlakyTest.dev vs TeamCity
FlakyTest.dev
flakytest.devTeamCity
jetbrains.comSide by side
| Figure | FlakyTest.dev | TeamCity |
|---|---|---|
| Price from | Free | $15/mo |
| Free plan | Yes | Yes |
| Free trial | Yes | Yes |
| Open source | — | — |
| Web | Yes | Yes |
| Windows | — | Yes |
| macOS | — | Yes |
| Linux | — | Yes |
| Android | — | Yes |
| API | — | Yes |
| Self-hosted | — | Yes |
| Plans published | 2 | 8 |
| Spec score · Test Flakiness Management Software | 7.6 (#19) | 8.7 (#8) |
| Headquarters | Ljubljana, Slovenia | Prague, Czech Republic |
| Integrations | The setup guide documents GitHub Actions configuration using a repository secret named FLAKYTEST_SECRET_TOKEN. | Built-in integrations use UI controls or specialized REST endpoints, and additional plugins can extend coverage. |
| Purpose | FlakyTest is a place to manage, monitor, and report on test suite performance. | TeamCity is a continuous integration and delivery server for automating builds, tests, and software delivery. |
| Support | The documentation directs users needing help, support, or feedback to email the maker. | The support page lists an issue tracker, support requests, documentation, and a community forum; Enterprise support offers 24/7 availability and guaranteed response times. |
Plans
TeamCity
- Professional (On-Premises)Free
- Cloud$15 a month
- TeamCity Cloud$15 a month
- Additional build credits$20, paid once
- Additional Cloud build credits$20, paid once
- Additional build agent$359 a year
- Additional On-Premises build agent$359 a year
- Enterprise (On-Premises)$2,399 a year
In their own words
FlakyTest.dev
FlakyTest.dev is a web service for managing, monitoring, and reporting on test suite performance, aimed at Python teams handling unreliable test suites. Its pytest plugin tracks tests and sends results when the FLAKYTEST_SECRET_TOKEN environment variable is set. Test ownership routes flaky-test notifications to the person who wrote the test. In the interface, users can mute a test without changing code or opening a pull request; the test keeps running and logging errors but does not fail the CI build. Each test has a run history, and failing tests show standard output, standard error, and other debug information. Run statistics help identify tests that slow builds. Flake detection and a quarantine workflow support unreliable-test management. The setup guide documents GitHub Actions configuration with a repository secret. The OSS plan is free for personal projects, non profits, and open source projects. Business includes a 60-day free trial, with later pricing undetermined.
TeamCity
TeamCity is a continuous integration and delivery server for automating software builds, tests and delivery. Build chains organize pipelines into reusable steps that can run in parallel and share results across configurations. Build caching, artifact dependencies and incremental builds reduce repeated work. TeamCity can test changes before commit and commit them automatically if they pass, while test-history analysis can flag inconsistent results. Pipeline settings can be kept in Kotlin DSL, XML or YAML, or generated from the interface. Supported version control systems include Git, Perforce, Mercurial, Subversion and CVS, with integrations for GitHub, GitLab, Bitbucket and Azure DevOps. Listed infrastructure integrations include AWS, Kubernetes, Google Cloud Platform, Azure Resource Manager and VMware vSphere. TeamCity is available on-premises or in the cloud. The Professional on-premises plan is free forever and includes three build agents, 100 build configurations and 10 pipelines. Cloud costs $15.00 per committer, per month, with a three-committer minimum; its trial is 14 days, while the on-premises Enterprise trial lasts 30 days.