Arroyo vs Confluent Cloud

Arroyo is an open-source distributed engine for processing stateful computations on data streams, with sub-second results. Users build streaming pipelines in analytical SQL, including complex queries, windows and joins. Arroyo supports checkpointing for fault tolerance and recovery, exactly-once processing, and horizontal and vertical rescaling; it says the engine can handle millions of events per second. Its connectors include Kafka, Kinesis, MQTT, MySQL, Postgres, Redis, Delta Lake, HTTP and WebSocket, among others. It reads and writes JSON, Avro, Parquet, text and binary, and includes more than 300 SQL functions. Rust scalar, aggregate and asynchronous user-defined functions can extend SQL. The web UI supports connection management, query development and testing, and pipeline monitoring; pipeline management is also available through a REST API. Single-node clusters are intended for development and testing, while production deployments use a distributed cluster with Arroyo’s scheduler or Kubernetes. It is Apache 2.0 licensed, free at free, and self-hostable.

Confluent Cloud is a managed data streaming platform built on a serverless Apache Kafka engine. It is designed for real-time streaming, event-driven applications, analytics, and AI/ML workloads. Cluster choices include Basic, Standard, Enterprise, and Freight, as well as Dedicated clusters with pre-provisioned resources. Confluent says its cloud-native engine adjusts cluster infrastructure to workload needs. Cluster Linking replicates, synchronizes, and shares data across regions, public clouds, and organizations. The service includes managed Kafka connectors for databases, data lakes, and warehouses, serverless processing with Apache Flink, and a managed Stream Governance suite. Tableflow can materialize Kafka topics and schemas as Apache Iceberg and Delta Lake tables. Listed security features include RBAC, self-managed encryption keys, client-side field-level encryption, and audit logs. Users can manage clusters and topics through a web UI or CLI; supported client languages include Java, Python, .NET, Go, Node.js, and C/C++. It is available on AWS, Azure, and Google Cloud across more than 100 regions. A free plan is available; paid plans start at $385/mo, and new signups receive a 30-day trial.