Skip to content
SPECIFICTION
Get listed

CircleCI vs FlakyTest.dev

CircleCI

circleci.com

FlakyTest.dev

flakytest.dev

Side by side

FigureCircleCIFlakyTest.dev
Price from$15/moFree
Free plan—Yes
Free trial—Yes
Open source——
WebYesYes
WindowsYes—
macOSYes—
LinuxYes—
APIYes—
Self-hostedYes—
Plans published32
Spec score · Test Flakiness Management Software8.1 (#9)7.6 (#19)
HeadquartersSan Francisco, California, United StatesLjubljana, Slovenia
SupportCommunity support is included across plans; ticket-based global support and 8x5 support SLAs are available on Performance and Scale, with 24x7 SLAs available on Scale.The documentation directs users needing help, support, or feedback to email the maker.

Plans

CircleCI

  • Performance$15 a month
  • FreePrice not published
  • ScalePriced on request

CircleCI pricing →

FlakyTest.dev

  • OSSFree
  • BusinessPriced on request

FlakyTest.dev pricing →

In their own words

CircleCI

CircleCI provides CI/CD tooling for engineering teams to ship code and validate changes. Smarter Testing includes test impact analysis, dynamic test splitting, and automatic reruns of failed tests. Execution options include Docker, Linux, Arm, Windows, macOS, and self-hosted runners. CircleCI Cloud supports repositories hosted on GitHub, GitLab, GitLab self-managed, Bitbucket Cloud, and Cursor Origin, which is in beta. Teams can use Orbs, shareable configuration packages, and private orbs are available across listed plans. CircleCI Deploys lets teams monitor incremental releases and roll back automatically. Security controls include encrypted source-code communication, restricted contexts, audit logging, isolated build sandboxes, and OpenID Connect support. CircleCI says it is SOC 2 Type II compliant and has a FedRAMP Tailored designation. Its API supports programmatic interaction with pipelines, projects, and organizations. A free plan is listed, and Performance starts at $15 a month. Free-plan credits expire after a month without rollover; paid credits roll over and expire after a year.

Visit circleci.com

FlakyTest.dev

FlakyTest.dev is a web service for managing, monitoring, and reporting on test suite performance, aimed at Python teams handling unreliable test suites. Its pytest plugin tracks tests and sends results when the FLAKYTEST_SECRET_TOKEN environment variable is set. Test ownership routes flaky-test notifications to the person who wrote the test. In the interface, users can mute a test without changing code or opening a pull request; the test keeps running and logging errors but does not fail the CI build. Each test has a run history, and failing tests show standard output, standard error, and other debug information. Run statistics help identify tests that slow builds. Flake detection and a quarantine workflow support unreliable-test management. The setup guide documents GitHub Actions configuration with a repository secret. The OSS plan is free for personal projects, non profits, and open source projects. Business includes a 60-day free trial, with later pricing undetermined.

Visit flakytest.dev