Amazon Kinesis Data Streams vs Apache Pulsar

Amazon Kinesis Data Streams is a cloud, serverless service for capturing, processing, and analyzing live data streams. It can continuously capture gigabytes per second from hundreds of thousands of sources, then make records available to multiple real-time analytics applications, Amazon Managed Service for Apache Flink, or AWS Lambda within 70 milliseconds of collection. Data is synchronously replicated across three Availability Zones in an AWS Region. Each stream supports up to 20 consumers with dedicated read throughput, and retention can be configured for as long as 365 days. Server-side encryption uses AWS Key Management Service keys, while Amazon VPC provides private access. Integrations include DynamoDB, Aurora, Database Migration Service, CloudWatch, Lambda, Managed Service for Apache Flink, and Kinesis Data Firehose. Records can be delivered to Amazon S3 Tables as Apache Iceberg tables or to S3 buckets in source format. It supports log and event streaming, real-time analytics, event-driven applications, and real-time AI or machine-learning workloads. Access is through an API or web interface. Pricing is paid from $0.03, with no free plan or AWS Free Tier availability.

Apache Pulsar is a free, open-source distributed messaging and streaming platform built for cloud use. It supports message consumption with less than 10 ms latency and up to one million unique topics in a cluster. Configurable replication across geographic regions and automatic client failover to healthy clusters support multi-region deployments. Tenants and policies control access to data and actions, and can isolate brokers to limit noisy-neighbor effects. Maintained client libraries are available for Java, Go, Python, C++, Node.js, and C#. Built-in connectors move data to and from systems including Kafka, RabbitMQ, MySQL, PostgreSQL, MongoDB, Cassandra, and Elasticsearch. Pulsar Functions process messages with user-defined logic in Java, Python, or Go. Tiered storage can offload data from BookKeeper to blob storage such as S3 for unlimited retention. It can run in Docker or Kubernetes, with a Helm chart for Kubernetes. Encryption, authentication, and authorization are available but are not enabled by default. The 5.0.0-M2 milestone is for early testing and feedback, not production use.