Streaming Throughput Benchmark

Streaming throughput benchmarking measures the performance of event-streaming platforms such as Apache Kafka, Apache Pulsar, Redpanda, and RabbitMQ. These systems sit at the center of modern data pipelines, ingesting and delivering high volumes of events between producers and consumers. The benchmark characterizes how much data they can move and how quickly, under realistic durability and concurrency settings — the questions that determine pipeline capacity and cost.

What It Measures

Key metrics are producer throughput (messages or megabytes published per second), consumer throughput (consumed per second, which can differ under fan-out), end-to-end latency (publish to delivery), and tail latency (p99/p999), which often matters more than the average for real-time pipelines. Throughput and latency are reported as a trade-off, since pushing throughput higher generally raises tail latency.

Methodology

The OpenMessaging Benchmark Framework is a common vendor-neutral harness; Kafka also ships kafka-producer-perf-test and kafka-consumer-perf-test. A run provisions a cluster, creates topics with a chosen partition count and replication factor, then drives producers at a target rate with a fixed message size while consumers read in parallel. Critical variables include message size, partition count, replication factor, producer acknowledgment level (acks), batching and compression settings, and whether producers run at peak or a fixed offered rate. The harness records sustained throughput and a full latency histogram. Durability settings have a large effect: requiring acknowledgment from all replicas trades throughput and latency for safety, so results must always state them.

How to Interpret Results

Always report message size, partition and replication counts, acks level, and batching/compression — these dominate the numbers and make undocumented results meaningless. Prefer fixed-rate runs that measure latency at a sustainable load over peak-throughput runs that ignore latency. Read p99/p999 latency, not just averages, for real-time use cases. Verify the consumers actually kept up with producers (no growing lag), since unbounded buffering can inflate apparent throughput. Compare platforms only under identical durability guarantees.

Limitations

Synthetic uniform messages do not capture real key distributions, consumer-group rebalancing, or mixed workloads. Results are extremely sensitive to configuration and hardware (disk and network especially), so cross-study comparisons are unreliable. Many vendor benchmarks tune settings to favor their product. Use streaming throughput benchmarks to size pipelines and compare platforms under documented, identical durability settings, and validate with your own message shapes and consumer topology. Because streaming platforms underpin so many pipelines, a carefully controlled throughput-and-latency benchmark is often the deciding factor when choosing between them or sizing a cluster.