Working with Apache Kafka

Apache Kafka is a real-time data pipeline processor. It s scalable, fault tolerance, has high execution speeds with fluid integrations are some of the key hallmarks that make Kafka an integral part of many Enterprise Data architectures. In this lab intensive two day course, you will learn how to use Kafka to build streaming solutions.

Introduction to Streaming Systems

• Fast data
• Streaming architecture
• Lambda architecture
• Message queues
• Streaming processors

Introduction to Kafka

• Architecture
• Comparing Kafka with other queue systems (JMS / MQ)
• Kaka concepts : Messages, Topics, Partitions, Brokers, Producers, and commit logs
• Kafka and Zookeeper
• Producing messages
• Consuming messages (Consumers and Consumer Groups)
• Message retention
• Scaling Kafka

Programming With Kafka

• Configuration parameters
• Producer API (Sending messages to Kafka)
• Consumer API (consuming messages from Kafka)
• Commits, Offsets, Seeking
• Schema with Avro

Kafka Streams

• Streams overview and architecture
• Streams use cases and comparison with other platforms
• Learning Kafka Streaming concepts (KStream, KTable, and KStore)
• KStreaming operations (transformations, filters, joins, and aggregations)

Administering Kafka

• Hardware/Software requirements
• Deploying Kafka
• Configuration of brokers/topics/partitions/producers/consumers
• Security: How secure Kafka cluster, and secure client communications (SASL and Kerberos)
• Monitoring: monitoring tools
• Capacity Planning: estimating usage and demand
• Trouble shooting: failure scenarios and recovery

Monitoring and Instrumenting Kafka

• Monitoring Kafka
• Instrumenting with Metrics library
• Instrument Kafka applications and monitor their performance

Before attending this course, you should:

• Be comfortable with Java
• Have experience working with databases
• Able to navigate the Linux command line
• Have basic knowledge of Linux editors (such as VI/nano) for editing code

Experienced Java Developers with database experience.