Apache Kafka for Event-Driven Spring Boot Microservices

Explore how a Kafka broker runs as a Kafka server on computer or cloud, forms a leader and followers cluster, and stores events in topic partitions for producers and consumers.

Explore how each partition of a topic has a leader to handle reads and writes, while followers replicate data for redundancy and fault tolerance, enabling balanced leadership across brokers.

Stop producers and consumers before shutting down Kafka servers to avoid message loss, then run the Kafka server stop CLI script for a graceful shutdown that closes log segments.

Learn to produce messages to a Kafka topic with the console producer, connect via bootstrap servers, and understand topic auto-creation versus preconfigured partitions and replicas.

Learn to send Kafka messages as key value pairs using the console producer, enabling key value support and a separator to preserve order for messages with the same key.

Learn how to consume key value pair messages from a Kafka topic by configuring the producer to send key and value, and using consumer options to print key and value.

Explore how a Kafka producer uses asynchronous communication to publish a user logged in event to a Kafka broker after authentication for analytics, keeping the login flow responsive.

Publish the product created event to a Kafka topic using Spring's Kafka Template after generating a unique product id with UUID and persisting product details, with options for asynchronous sending.

Explore asynchronous sending of a product created event to a Kafka topic, verify with a console consumer, and observe log messages showing the send flow.

Publish events to a Kafka topic synchronously by awaiting broker acknowledgement to ensure persistence, while comparing approaches using CompletableFuture join and get to toggle between async and sync.

Update the create product rest controller to handle exceptions by returning a custom json error object with a timestamp and message, using a logger and Spring's automatic json conversion.

Demonstrates synchronous publishing of a product created event to Kafka, verifies the producer and a consumer script, and confirms topic, partition, and offset metadata.

Configure the Spring Kafka producer to wait for acknowledgements from all in-sync replicas using spring.kafka.producer.acks to strengthen durability and prevent message loss.

Explore how Kafka producer retries and retry backoff work under broker failures, with two in-sync replicas, three brokers, and ten retries at a one second time interval.

Explore delivery timeout, request timeout, and linger, to optimize producer behavior; delivery timeout governs total send duration across replicas, including retries, while request timeout caps a single broker response.

Configure a Kafka producer in Java code by building a properties map, using a producer factory and a Kafka template, with precedence over application.properties for bootstrap servers.

Enable idempotence for the Kafka producer using the application.properties file, setting enable.idempotence, acks to all, and configuring retries and max in-flight requests per connection to five.

Enable kafka producer idempotence in a spring bean by configuring producer properties in code and loading from application properties, with acks all and max in flight capped at five.

Create a new Spring Boot project for an email notification microservice using Spring Tool Suite, Spring Starter Project, and Apache Kafka dependencies to act as a Kafka consumer.

Configure the Kafka consumer using the application properties file, specifying bootstrap servers, key and value deserializers, and a consumer group id for product created events.

Create a product created event handler class as a Spring component with a handle method annotated as a Kafka listener to process product created events from topic and log arrival.

Configure the Kafka consumer in Java code via a Spring bean and a Kafka consumer configuration class, reading bootstrap servers, deserializers, and group id from application properties.

Create a Kafka listener container factory using the bean annotation and the configured Kafka consumer factory, enabling listeners to receive messages and invoke their methods within a Spring Boot application.

Test the Kafka consumer bean configuration by launching microservices, sending a product creation request, and verifying messages flow to the email notification service.

Configure your Kafka consumer to recover from deserialization errors when reading product created events, preventing endless retries of faulty messages and ensuring the email notification service continues with subsequent messages.

Learn how a deserialization error halts a Kafka consumer when invalid json is published, and preview recovering with an error handling deserializer class in a Spring Boot microservice.

Demonstrate how the error handling deserializer helps a kafka consumer recover from deserialization errors in a spring boot microservice setup. See one-time errors recover and new messages process correctly.