Chaos Toolkit vs Reliably

Chaos Toolkit is a free, open-source chaos engineering toolkit for developers exploring systems to uncover weaknesses. Experiments define a steady-state hypothesis and probes to check it, along with a method of actions and probes. If the system does not meet the stated steady state, the method is not applied and the experiment must stop. Experiments can be declared in JSON or YAML files, making them suitable for collaboration and orchestration alongside code. The toolkit extends to systems through its Open API and can be embedded in a CI/CD chain. Listed integrations include AWS, Azure, Google Cloud Platform, Kubernetes, Kafka, Slack, Datadog, Grafana, OpenTelemetry, and Prometheus. The standalone CLI is available under Apache License 2.0. It is implemented in Python 3, officially supports Python 3.8 and later, and has only been tested against CPython. Installation instructions cover macOS, Debian or Ubuntu, and Windows; users can install extensions after the CLI and core library. Reliably is a separate platform with SaaS, private SaaS, and on-premises options.

Reliably helps teams examine operational risks and incidents as part of resilience engineering. Built around an open-source chaos engineering platform based on Chaos Toolkit, it offers an experiment builder with more than 300 actions, including attacks and probes. Teams can import JSON Chaos Toolkit experiments, reuse templates, schedule runs, and review scores, freshness indicators, timelines, and dashboards. Experiments can run as GitHub Actions, through Reliably-managed execution, with the CLI, in Docker, or as Kubernetes jobs or cron jobs. Listed targets include AWS, Azure, Google Cloud, Kubernetes, Prometheus, and Spring. Pre-checks can prevent a run from starting; configured safeguards can interrupt one during execution. Environments store variables and encrypted secrets for the app, CLI, or GitHub workflows. The Free plan is free and limits teams to one member, 180 cloud execution minutes per month, a 10-minute cloud execution cap, and two weeks of data retention. Start costs £40 a month, billed monthly.