Google Cloud Datastream vs pgstream

Google Cloud Datastream is a serverless change data capture and replication service for synchronizing data. Streams can capture ongoing updates and backfill historical records, while the service scales automatically without resources for users to provision or manage. Supported destinations include BigQuery, Cloud Storage, and Apache Iceberg tables. Documented sources include Oracle, MySQL, PostgreSQL, AlloyDB, SQL Server, MongoDB, Spanner, Salesforce, ServiceNow, Workday, and Microsoft Dataverse; some sources are marked as preview. Datastream can work with Dataflow templates to load data from Cloud Storage into destinations including BigQuery, Cloud SQL, and Spanner. Connectivity options include IP allowlisting, a forward SSH tunnel, and private connectivity over VPC peering. Processing is charged, idle streams incur no cost, and other Google Cloud resources are billed separately. The free tier includes 100 GiB of eligible CDC data per month per billing account. Events are delivered at least once, but ordering is not guaranteed. Datastream is administered through the Google Cloud console, CLI, or REST API.

pgstream is a free, open-source tool for moving PostgreSQL data and schema changes to other systems. Run it as a command-line program or embed it as a Go library; it is self-hosted and requires PostgreSQL. Destinations include PostgreSQL, Elasticsearch or OpenSearch, Kafka, and webhooks. It can take an initial or on-demand snapshot for bulk loads and point-in-time copies, and tracks schema changes so receiving systems can reflect table updates. During replication, column values can be transformed with greenmask, neosync, go-masker, or custom transformers, including to mask production data headed for development or staging. Kafka support includes schema-based partitioning. Installation options include a Go binary, building from source, and Homebrew on macOS or Linux. The project uses the Apache License 2.0 and costs nothing. Its initial release has constraints: one Kafka topic, the wal2json PostgreSQL plugin, no row-level filtering, and a primary-key or unique non-null column requirement. Community questions and issues go through GitHub issues.