A software developer walks us through an imaginary use case in order to demonstrate developing data streaming applications using microservice architecture.
Let’s be friends:
Comment ( 0)
Join the DZone community and get the full member experience.
With Hortonworks DataFlow (HDF) 3.3 now supporting Kafka Streams, we are truly excited about the possibilities of the applications that you can benefit from when combined with the rest of our platform.
In this post, we will demonstrate how Kafka Streams can be integrated with Schema Registry, Atlas, and Ranger to build a set of microservices apps for a fictitious use case.
For this sample application, a trucking fleet company has sensors outfitted on their fleet of trucks. The geo-event sensor captures important events from the truck (e.g: lane change, breaking, start/stop, acceleration along with its geolocation) and the speed sensor captures the speed of the truck at different intervals. These two sensors are streaming data into their own respective Kafka topics: syndicate-geo-event-avro and syndicate-speed-event-avro. The requirements for the use case are the following:
In the past, we have shown how to use Streaming Analytics Manager (SAM) to implement these requirements. With Kafka Streams, you can implement these requirements with a set of light-weight microservices that are highly decoupled and independently scalable. The below diagram illustrates this architecture:
Kafka Streams has rich support for joins and provides compositionally simple APIs to do stream-to-stream joins and stream-to-table joins using the KStream and KTable abstractions.
