AWS Entity Resolution vs Entify

AWS Entity Resolution links records that may describe the same customer, product, business, or healthcare entity across applications, channels, and data stores. It offers rule-based, machine learning, and data service provider matching. Rule-based users can adjust input fields, priorities, and rules; the preconfigured machine learning model compares consumer fields such as names, email addresses, phone numbers, addresses, and birth dates, then assigns match groups confidence scores from 0.0 to 1.0. Inputs come from AWS Glue, with a limit of 20. Rule-based matching can process new Amazon S3 data incrementally, and the GetMatchId API can synchronously look up an existing match ID while hashing personally identifiable attributes. Access is through the AWS console and API. Personally identifiable data is encrypted at rest by default with AWS-owned keys, with customer-managed symmetric AWS KMS keys also supported. Provider matching includes LiveRamp, TransUnion, and Unified ID 2.0 identifiers, subject to a provider license agreement. AWS states that the service meets PCI DSS and SOC 1, SOC 2, and SOC 3 measures. Pricing is paid, from $0.10; there is no free plan.

Entify is a free, open-source workspace for linking records, resolving entities, and deduplicating datasets. It uses Splink 4 and DuckDB for matching, with exact, Jaro-Winkler, Jaccard, and Levenshtein comparisons. Automatic setup estimates column types, proposes blocking rules, and explains its choices for users to adjust. It reads CSV, TSV, Parquet, JSON, and Excel files, and reports row counts, distinct values, and completeness by column. Review tools include match evidence, score and threshold analysis, and PDF audit reports. Merge exports keep one surviving record per entity and retain source IDs for tracing or undoing merges. Entify runs locally with Docker or with Python 3.11+ and Node 20+; demo mode works without optional sign-in or shared storage. Its Docker quickstart needs no accounts, API keys, or external services, and uploaded data stays on the machine. The project is intended for data stewards and teams without specialist record-linkage expertise. Optional semantic blocking requires an extra install of roughly 600 MB of Torch dependencies.