Apache Arrow DataFusion vs Trino

Apache Arrow DataFusion is a free, extensible query engine written in Rust that uses Apache Arrow as its in-memory format. It is intended for developers building database and analytics systems tailored to particular workloads, rather than as a ready-made hosted database. Queries run in-process, using threads for parallel execution. DataFusion offers SQL and DataFrame APIs, while related subprojects provide Python and Java interfaces. Documented capabilities include SQL parsing and planning, parallel and streaming execution, and query optimization. Built-in data sources include CSV, Parquet, JSON, Avro, and Arrow. Developers can add sources through the TableProvider trait and customize functions, operators, and other components. Runtime features include enforced memory limits and disk spilling for sorts, grouping, and joins. The core project is designed for in-process use; Ballista is a related distributed processing extension. Rust users commonly add DataFusion from crates.io, while official Apache releases are source artifacts. The project recommends verifying release artifacts with an OpenPGP signature or SHA-512 checksum.

Trino is a distributed SQL query engine for analytics across large datasets held in different systems. It supports ANSI SQL and can query multiple sources together, such as joining S3 object storage data with MySQL in one query. Its parallel, distributed design is intended for efficient, low-latency analytics and data warehousing, not general-purpose relational database use or OLTP workloads. Connectors cover systems including BigQuery, Cassandra, ClickHouse, Elasticsearch, MongoDB, MySQL, Oracle, PostgreSQL, Redis, Snowflake, and SQL Server. Project-maintained client options include JDBC, Go, JavaScript, Python, and C#, alongside a command-line interface and Grafana data-source plugin. Trino runs in Kubernetes or Docker, or as a manually deployed server, and is optimized for cloud and on-premises environments including Amazon, Azure, and Google Cloud. AI functions support OpenAI and Anthropic directly and other models through Ollama, with the language model supplied as an external service. Trino is free under the Apache License 2.0 as self-managed software. It supports multiple authentication and access-control options, but a default installation has no security features enabled.