dplyr vs Ibis

dplyr is a free R package for transforming data frames and tibbles through a set of code-based operations. Its main verbs add variables with mutate(), choose variables with select(), filter cases with filter(), summarize values with summarise(), and order rows with arrange(). Combine them with group_by() to apply operations to groups, or use two-table verbs to work with pairs of datasets. The across() helper repeats an operation over several columns, while rowwise() supports row-by-row work. The programming guide covers data masking, tidy selection, and use in functions or loops. For other data settings, dbplyr translates dplyr code into SQL for relational databases; the documentation also lists Arrow, dtplyr, duckplyr, and sparklyr as alternative backends for larger datasets. dplyr runs on Linux, macOS, and Windows and is licensed under MIT. The CRAN package requires R 4.1.0 or later, and installation instructions cover CRAN and a development version from GitHub. The project recommends dplyr to people learning data transformation in R for Data Science.

Ibis is a free, open-source Python dataframe library that provides a consistent API across more than 20 query backends. It lets users compose SQL and Python dataframe code, bringing data engineering and data science workflows together. For local dataframes, Ibis uses DuckDB by default and also names Polars and DataFusion. The project describes a workflow that can start locally and move to remote execution by changing a single line of code.

Ibis supports and tests Google BigQuery, Snowflake, Databricks Cloud, and Amazon Athena; support for those cloud backends could move to best effort if funding ends. Supported data formats include CSV, TSV, JSON, Parquet, Delta Lake, and Avro. Install Ibis with a supported backend using pip, conda, mamba, or pixi. One package distinction matters: the PyPI package named ibis is separate from ibis-framework, and the two cannot coexist in one Python environment because both use the ibis module name. Operation coverage varies by backend due to differences in SQL dialects and upstream support. The project recommends GitHub and Zulip for help, and Voltron Data offers commercial support.