Amazon Kinesis Data Streams vs Apache Kafka

Amazon Kinesis Data Streams is a cloud, serverless service for capturing, processing, and analyzing live data streams. It can continuously capture gigabytes per second from hundreds of thousands of sources, then make records available to multiple real-time analytics applications, Amazon Managed Service for Apache Flink, or AWS Lambda within 70 milliseconds of collection. Data is synchronously replicated across three Availability Zones in an AWS Region. Each stream supports up to 20 consumers with dedicated read throughput, and retention can be configured for as long as 365 days. Server-side encryption uses AWS Key Management Service keys, while Amazon VPC provides private access. Integrations include DynamoDB, Aurora, Database Migration Service, CloudWatch, Lambda, Managed Service for Apache Flink, and Kinesis Data Firehose. Records can be delivered to Amazon S3 Tables as Apache Iceberg tables or to S3 buckets in source format. It supports log and event streaming, real-time analytics, event-driven applications, and real-time AI or machine-learning workloads. Access is through an API or web interface. Pricing is paid from $0.03, with no free plan or AWS Free Tier availability.

Apache Kafka is an open-source distributed event-streaming platform for data pipelines, streaming analytics, data integration, and mission-critical applications. Applications can publish and subscribe to event streams, retain them durably, and process them either in real time or later. Kafka supports durable, fault-tolerant replication across availability zones, data centers, or geographic regions. Kafka Streams provides stateful operations, aggregations, windowing, joins, event-time processing, and exactly-once processing. Kafka Connect links hundreds of event sources and sinks, including Postgres, JMS, Elasticsearch, and AWS S3. Deployments can use bare metal, virtual machines, or containers, on premises or in the cloud, with self-managed or vendor-managed options. The project describes network-limited throughput, latency as low as 2 ms, and clusters of up to 1,000 brokers. Kafka is self-hosted, free, and open source under Apache License 2.0. A local quickstart requires Java 17 or later.