Apache Arrow DataFusion vs Azure Synapse Analytics
Apache Arrow DataFusion
datafusion.apache.org
Azure Synapse Analytics
azure.microsoft.comSide by side
| Figure | Apache Arrow DataFusion | Azure Synapse Analytics |
|---|---|---|
| Price from | Free | — |
| Free plan | Yes | — |
| Free trial | — | — |
| Open source | — | — |
| Web | — | Yes |
| macOS | Yes | — |
| Linux | Yes | — |
| Self-hosted | Yes | — |
| Plans published | 1 | 7 |
| Spec score · Query Engine Software | 8.0 (#10) | 6.2 (#21) |
Plans
Azure Synapse Analytics
- Serverless SQL poolPriced on request
- Dedicated SQL pool — pay as you goPriced on request
- Dedicated SQL pool — 1-year reservedPriced on request
- Dedicated SQL pool — 3-year reservedPriced on request
- Apache Spark — Memory OptimizedPriced on request
- Apache Spark — GPU acceleratedPriced on request
- Data Explorer computePriced on request
In their own words
Apache Arrow DataFusion
Apache Arrow DataFusion is a free, extensible query engine written in Rust that uses Apache Arrow as its in-memory format. It is intended for developers building database and analytics systems tailored to particular workloads, rather than as a ready-made hosted database. Queries run in-process, using threads for parallel execution. DataFusion offers SQL and DataFrame APIs, while related subprojects provide Python and Java interfaces. Documented capabilities include SQL parsing and planning, parallel and streaming execution, and query optimization. Built-in data sources include CSV, Parquet, JSON, Avro, and Arrow. Developers can add sources through the TableProvider trait and customize functions, operators, and other components. Runtime features include enforced memory limits and disk spilling for sorts, grouping, and joins. The core project is designed for in-process use; Ballista is a related distributed processing extension. Rust users commonly add DataFusion from crates.io, while official Apache releases are source artifacts. The project recommends verifying release artifacts with an OpenPGP signature or SHA-512 checksum.
Azure Synapse Analytics
Azure Synapse Analytics has no write-up on Specifiction yet; its figures are above.