dedupe vs Entify
dedupe
github.comEntify
github.comSide by side
| Figure | dedupe | Entify |
|---|---|---|
| Price from | — | Free |
| Free plan | — | Yes |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| Windows | Yes | — |
| macOS | Yes | — |
| Linux | Yes | — |
| API | Yes | Yes |
| Self-hosted | Yes | Yes |
| Plans published | 0 | 1 |
| Spec score · Data Deduplication Software | 7.9 (#12) | 8.4 (#2) |
| Data formats | The former service’s FAQ says uploads accepted Excel spreadsheets and CSV files, with one record per row and column names in the first row. | It reads CSV, TSV, Parquet, JSON, and Excel files. |
| Integrations | The project describes dedupe as a library that can be integrated into applications or import scripts; it also documents CSV deduplication through the separate csvdedupe command-line tool. | Optional sign-in and shared storage use Clerk authentication and Supabase persistence. |
| Intended users | The maintainers describe the library as suitable for integration into applications or import scripts, while the former Dedupe.io service was designed as an online tool for non-developers. | The project describes itself as intended to make record linkage usable by data stewards and teams without specialist record-linkage expertise. |
| License | The GitHub repository identifies the project as MIT licensed. | The repository lists an MIT license. |
Plans
In their own words
Entify
Entify is a free, open-source workspace for linking records, resolving entities, and deduplicating datasets. It uses Splink 4 and DuckDB for matching, with exact, Jaro-Winkler, Jaccard, and Levenshtein comparisons. Automatic setup estimates column types, proposes blocking rules, and explains its choices for users to adjust. It reads CSV, TSV, Parquet, JSON, and Excel files, and reports row counts, distinct values, and completeness by column. Review tools include match evidence, score and threshold analysis, and PDF audit reports. Merge exports keep one surviving record per entity and retain source IDs for tracing or undoing merges. Entify runs locally with Docker or with Python 3.11+ and Node 20+; demo mode works without optional sign-in or shared storage. Its Docker quickstart needs no accounts, API keys, or external services, and uploaded data stays on the machine. The project is intended for data stewards and teams without specialist record-linkage expertise. Optional semantic blocking requires an extra install of roughly 600 MB of Torch dependencies.