Kafka
21 items tagged with "kafka"
Best Practices3
Schema Evolution and Schema Registry
Managing how data schemas change over time with compatibility rules and a central registry so producers and consumers evolve without breaking each other.
Apache Kafka Streaming Best Practices
Design and operational guidance for building reliable, scalable event streaming on Apache Kafka, covering topics, partitions, delivery semantics, and consumers.
AsyncAPI Specification
A standard, machine-readable format for describing event-driven and message-based APIs across protocols like Kafka, MQTT, and AMQP, analogous to OpenAPI for REST.
Tutorials5
Building Event-Driven Systems with Kafka
Create scalable event-driven architectures using Apache Kafka
How to stream database changes with Debezium CDC
Capture row-level changes from PostgreSQL using Debezium and Kafka Connect, producing an ordered stream of insert, update, and delete events.
How to build a Kafka producer and consumer
Create a Kafka topic, produce messages with keys, and consume them in a group, covering offsets, partitions, and delivery basics.
How to load Kafka topics into a database with Kafka Connect
Use a Kafka Connect sink connector to stream topic data into a relational database with no custom consumer code.
How to use Avro and Schema Registry with Kafka
Define an Avro schema, register it, and produce and consume typed Kafka records with schema evolution and compatibility checks.
Reference Architectures4
Event-Driven Microservices
Architecture pattern for building loosely-coupled microservices using event sourcing and CQRS
Event-Driven Microservices on Kubernetes
A Kubernetes-native reference design for loosely coupled microservices that communicate through Kafka events with service-level autoscaling.
Real-Time Streaming Platform with Kafka
A real-time streaming platform on Kafka with stream processing, a schema registry, and exactly-once pipelines on Kubernetes.
Change Data Capture Pipeline with Debezium
A CDC pipeline streaming row-level changes from operational databases into a warehouse using Debezium and Kafka on AWS.
Stacks7
Kafka + Flink Streaming Stack
Real-time stream processing stack pairing Apache Kafka for durable event streams with Apache Flink for stateful, low-latency computation.
Event-Driven Microservices Stack
Asynchronous microservices architecture using Kafka as an event backbone with independently deployable services for loose coupling and scalability.
Kafka + Flink + Iceberg Streaming Stack
Real-time streaming architecture: Kafka transports events, Flink processes them with stateful low-latency compute, and Iceberg lands them in an open lakehouse.
ClickHouse Real-Time Analytics Stack
High-performance analytics stack: ClickHouse ingests event streams from Kafka and serves sub-second OLAP queries powering user-facing dashboards.
Kafka + ksqlDB
A stream-processing stack using Apache Kafka for event transport and ksqlDB for SQL-based streaming transformations and materialized views.
Apache Pinot Real-Time Analytics
A user-facing analytics stack built on Apache Pinot for ultra-low-latency, high-throughput queries over fresh streaming data.
Kafka + Flink Streaming Lakehouse
An end-to-end streaming lakehouse stack: Kafka ingests events, Flink processes them in real time, and Iceberg tables on S3 serve analytics.