Skip to content
SPECIFICTION
Get listed

Apache Arrow DataFusion

datafusion.apache.org

Free plan macOSLinuxSelf-hosted
Visit datafusion.apache.org
Spec score 8.0 #10 of 28 in Query Engine Software

What it is

Apache Arrow DataFusion is a free, extensible query engine written in Rust that uses Apache Arrow as its in-memory format. It is intended for developers building database and analytics systems tailored to particular workloads, rather than as a ready-made hosted database. Queries run in-process, using threads for parallel execution. DataFusion offers SQL and DataFrame APIs, while related subprojects provide Python and Java interfaces. Documented capabilities include SQL parsing and planning, parallel and streaming execution, and query optimization. Built-in data sources include CSV, Parquet, JSON, Avro, and Arrow. Developers can add sources through the TableProvider trait and customize functions, operators, and other components. Runtime features include enforced memory limits and disk spilling for sorts, grouping, and joins. The core project is designed for in-process use; Ballista is a related distributed processing extension. Rust users commonly add DataFusion from crates.io, while official Apache releases are source artifacts. The project recommends verifying release artifacts with an OpenPGP signature or SHA-512 checksum.

Spec sheet

Product
Apache DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format.Read 29 Sep 2026 · datafusion.apache.org
Intended users
The core project provides libraries and binaries for developers building database and analytics systems customized to particular workloads.Read 29 Sep 2026 · datafusion.apache.org
Support
The project directs users to its community communication channels for getting in touch.Read 29 Sep 2026 · datafusion.apache.org
APIs
DataFusion offers SQL and DataFrame APIs, and related subprojects provide Python and Java interfaces.Read 29 Sep 2026 · datafusion.apache.org
Distribution
The core DataFusion project is designed for in-process use; Ballista is a related distributed processing extension.Read 29 Sep 2026 · datafusion.apache.org
Downloads
Rust users commonly add DataFusion from crates.io, while official Apache releases are provided as source artifacts.Read 29 Sep 2026 · datafusion.apache.org
Execution
DataFusion runs queries in-process using threads for parallel query execution.Read 29 Sep 2026 · datafusion.apache.org
Extensibility
Developers can add data sources through the TableProvider trait and customize functions, operators, and other components.Read 29 Sep 2026 · datafusion.apache.org
Formats
Built-in data source support includes CSV, Parquet, JSON, Avro, and Arrow.Read 29 Sep 2026 · datafusion.apache.org
Governance
The project is governed through the Apache Software Foundation process.Read 29 Sep 2026 · datafusion.apache.org
Query features
Documented features include SQL parsing and planning, parallel and streaming execution, and query optimization.Read 29 Sep 2026 · datafusion.apache.org
Release verification
The download page recommends verifying release artifacts with an OpenPGP signature or SHA-512 checksum.Read 29 Sep 2026 · datafusion.apache.org
Runtime limits
Documented runtime features include enforced memory limits and disk spilling for sorts, grouping, and joins.Read 29 Sep 2026 · datafusion.apache.org

Plans and prices

Apache DataFusion Free Open source project; official releases are source artifacts; distributed as a Rust library and CLI Read 29 Sep 2026 · datafusion.apache.org

Every Apache Arrow DataFusion plan and price →

Strong points and limits

Strong points

  • Free open-source project.
  • Provides SQL and DataFrame APIs.
  • Built-in sources include CSV, Parquet, JSON, Avro, and Arrow.
  • Supports parallel and streaming execution.
  • Includes memory limits and disk spilling.

Limits

  • Core project is designed for in-process use.
  • Official Apache releases are source artifacts.
  • Distributed processing is a related extension.

Who it suits

DataFusion suits developers building custom database or analytics systems who want an embeddable query engine and can work with a Rust library or CLI. Teams seeking distributed processing can look at the related Ballista extension.

Verdict

DataFusion provides query planning, execution, and extensibility for developers embedding analytics capabilities in their own systems. Its in-process design and source-artifact releases are worth considering when planning deployment.

Where Specifiction places it

Alternatives to Apache Arrow DataFusionAll alternatives →