Apache Kafka Streams

What is Apache Kafka Streams?

Apache Kafka Stream is a collection of client library which is used for building applications. It is specifically used for the applications which transform input Kafka topics into output Kafka topics. It offers the user to write the application in Java and Scala and take the benefit of Apache Kafka cluster technology. Kafka stream processing is similar to standard computer programming.


Advantage of Apache Kafka Streams

Apache Kafka streaming provides the following advantages.

  • Apache Kafka streams are highly scalable, fault-tolerant, and flexible.
  • We can install it on VMs, cloud systems, and containers.
  • Apache Stream easily integrates with Kafka security.
  • Kafka streaming provides a rich set of APIs to write standard Java and Scala applications.
  • There is no separate processing cluster required for Apache Kafka streaming.
  • We can easily install Apache Kafka streaming on different platforms such as windows, mac, and Linux.

Apache Kafka Streams use cases

Let us see some use cases of Apache Kafka Streams.

  • The New York Times uses Apache Kafka and Kafka Streams to send near real-time news content to the different application which publishes it further.
  • Pinterest uses Apache Kafka and Kafka Streams to perform the predictive budgeting.
  • Zalando uses Kafka as an ESB (Enterprise Service Bus) to perform near-real-time business intelligence.
  • Rabobank uses Apache Kafka to alerts customers in real-time upon financial events and is built using Kafka Streams.
  • LINE uses Apache Kafka as a central datahub to communicate with one another.

Apache Kafka Stream Processing Topology

Let us see Apache Kafka Stream Processing Topology.

1. Stream

A stream is a continous,fault-tolerant form of data set that is the main abstraction of Apache Kafka stream. The data inside the stream is represented as a key and value pair.

2. Stream Processing Application

The user can use the Kafka Streams library to process the application.

3. Stream Processor

Apache Kafka Stream Processor receives the input records from its upstream topology and processes it and after that, it transfers data to the stream.

There are two main processors present in this topology.

    3.1 Source Processor

    Apache Kafka receive uses the Kafka topics to receive the input stream and then it forwards it to the downstream processors.

    3.2 Sink Processor

    Apache Kafka Sink Processor receives the records from upstream to the Kafka topics.