Amazon Managed Streaming for Apache Kafka vs Apache Pulsar

Amazon Managed Streaming for Apache Kafka (MSK) manages Apache Kafka infrastructure on AWS for developers and platform engineers. It provisions and configures clusters, replaces failed servers, coordinates patches and upgrades, and handles monitoring and scaling. Provisioned clusters let customers choose broker and storage options, while Serverless clusters adjust resources to workload needs. MSK keeps compatibility with the Apache Kafka client protocol, so existing Kafka applications and tools can work without application code changes. Integrations include Amazon S3, Redshift, Managed Service for Apache Flink, Lambda, Glue Schema Registry, and Database Migration Service. Clusters span multiple Availability Zones, with three as the default, and the service-level agreement specifies three 9s of availability. Security includes IAM, SASL/SCRAM or mutual TLS for Provisioned clusters, and encryption in transit and at rest. CloudWatch supplies over 50 default metrics at no additional cost. MSK is paid, with separate charges possible for storage, processing, connectivity, and enhanced metrics.

Apache Pulsar is a free, open-source distributed messaging and streaming platform built for cloud use. It supports message consumption with less than 10 ms latency and up to one million unique topics in a cluster. Configurable replication across geographic regions and automatic client failover to healthy clusters support multi-region deployments. Tenants and policies control access to data and actions, and can isolate brokers to limit noisy-neighbor effects. Maintained client libraries are available for Java, Go, Python, C++, Node.js, and C#. Built-in connectors move data to and from systems including Kafka, RabbitMQ, MySQL, PostgreSQL, MongoDB, Cassandra, and Elasticsearch. Pulsar Functions process messages with user-defined logic in Java, Python, or Go. Tiered storage can offload data from BookKeeper to blob storage such as S3 for unlimited retention. It can run in Docker or Kubernetes, with a Helm chart for Kubernetes. Encryption, authentication, and authorization are available but are not enabled by default. The 5.0.0-M2 milestone is for early testing and feedback, not production use.