Amazon Deequ vs Sqitch

Amazon Deequ is an Apache Spark library for writing data unit tests that measure quality in large datasets, including datasets with billions of rows. Its checks cover row counts, completeness, uniqueness, allowed values, nonnegative values, patterns, and approximate quantiles. Deequ also supports data profiling, stored and queryable metrics, anomaly detection over time, automatic constraint suggestions, and incremental metric computation. Its declarative Data Quality Definition Language includes rules for counts, completeness, uniqueness, statistics, schema matching, freshness, and custom SQL. Row-level evaluation can identify records that pass or fail supported rules, while dataset-level rules such as RowCount and Mean are skipped in that mode. Deequ targets specific Apache Spark versions, so the release must match the Spark version; versions 2.1.0 and later require Java 11. The project provides Maven and sbt dependency examples. It is licensed under Apache 2.0, and Python users can use the PyDeequ interface.

Sqitch is a database change management application for deploying and tracking database changes. It is a standalone system, not tied to a framework, ORM, database engine, or development environment. Changes are written as scripts native to the chosen database, and can declare dependencies—including changes in other Sqitch projects—to establish execution order. A plan file and Merkle tree pattern support deployment integrity without requiring numbered change names. Sqitch can deploy and revert changes, report status and history, and run optional verification scripts. Its manual lists PostgreSQL, YugabyteDB, CockroachDB, SQLite, MySQL, MariaDB, Oracle, Firebird, Vertica, Exasol, Snowflake, and ClickHouse. Connecting and executing scripts requires the database's native command-line client and connector libraries. Installation options include source, Docker, Linux distribution packages, macOS Homebrew, and Windows instructions. Sqitch is free, with a license permitting use and distribution, and is available for Linux, macOS, Windows, and self-hosted use.