Apache Kafka Cookbook
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers, and more
Why Subscribe?
Free Access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Getting ready
How to do it…
How it works…
There's more…
See also
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Initiating Kafka
Introduction
Setting up multiple Kafka brokers
Getting ready
How to do it...
How it works…
There's more…
See also
Creating topics
Getting ready
How to do it...
How it works...
There's more…
Sending some messages from the console
Getting ready
How to do it...
How it works...
There's more…
Consuming from the console
Getting ready
How to do it...
How it works...
There's more...
2. Configuring Brokers
Introduction
Configuring the basic settings
Getting ready
How to do it...
How it works…
There's more…
Configuring threads and performance
Getting ready
How to do it...
How it works…
There's more...
Configuring the log settings
Getting ready
How to do it...
How it works…
There's more…
See also
Configuring the replica settings
Getting ready
How to do it...
How it works…
There's more...
Configuring the ZooKeeper settings
Getting ready
How to do it…
How it works…
See also
Configuring other miscellaneous parameters
Getting ready
How to do it...
How it works…
See also
3. Configuring a Producer and Consumer
Introduction
Configuring the basic settings for producer
Getting ready
How to do it...
How it works…
Configuring the thread and performance for producer
Getting ready
How to do it...
How it works…
See also
Configuring the basic settings for consumer
Getting ready
How to do it...
How it works…
Configuring the thread and performance for consumer
Getting ready
How to do it...
How it works…
Configuring the log settings for consumer
Getting ready
How to do it...
How it works…
Configuring the ZooKeeper settings for consumer
Getting ready
How to do it...
How it works…
Other configurations for consumer
Getting ready
How to do it...
How it works…
See also
4. Managing Kafka
Introduction
Consumer offset checker
Getting ready
How to do it...
How it works…
Understanding dump log segments
Getting ready
How to do it…
How it works…
Exporting the ZooKeeper offsets
Getting ready
How to do it...
How it works…
Importing the ZooKeeper offsets
Getting ready
How to do it...
How it works…
Using GetOffsetShell
Getting ready
How to do it…
How it works…
Using the JMX tool
Getting ready
How to do it...
How it works…
There's more…
Using the Kafka migration tool
Getting ready
How to do it...
How it works…
The MirrorMaker tool
Getting ready
How to do it...
How it works…
See also
Replay Log Producer
Getting ready
How to do it...
How it works…
Simple Consumer Shell
Getting ready
How to do it...
How it works…
State Change Log Merger
Getting ready
How to do it...
How it works…
Updating offsets in Zookeeper
Getting ready
How to do it...
How it works…
Verifying consumer rebalance
Getting ready
How to do it...
How it works…
5. Integrating Kafka with Java
Introduction
Writing a simple producer
Getting ready
How to do it...
How it works…
See also
Writing a simple consumer
Getting ready
How to do it...
How it works…
See also
Writing a high-level consumer
Getting ready
How to do it...
How it works…
See also
Writing a producer with message partitioning
Getting ready
How to do it...
How it works…
There's more…
Multithreaded consumers in Kafka
Getting ready
How to do it...
How it works…
6. Operating Kafka
Introduction
Adding and removing topics
Getting ready
How to do it...
How it works…
There's more…
See also
Modifying topics
Getting ready
How to do it...
How it works…
There's more…
Implementing a graceful shutdown
Getting ready
How to do it...
How it works…
Balancing leadership
Getting ready
How to do it...
How it works…
There's more…
Mirroring data between Kafka clusters
Getting ready
How to do it...
How it works…
There's more…
Expanding clusters
Getting ready
How to do it...
How it works…
There's more…
Increasing the replication factor
Getting ready
How to do it...
How it works…
There's more…
Checking the consumer position
Getting ready
How to do it...
How it works…
Decommissioning brokers
Getting ready
How to do it...
How it works…
7. Integrating Kafka with Third-Party Platforms
Introduction
Using Flume
Getting ready
How to do it...
How it works…
See also
Using Gobblin
Getting ready
How to do it...
How it works…
See also
Using Logstash
Getting ready
How to do it...
How it works…
There's more…
See also
Configuring Kafka for real-time
Getting ready
How to do it...
How it works…
Integrating Spark with Kafka
Getting ready
How to do it...
How it works…
There's more…
Integrating Storm with Kafka
Getting ready
How to do it...
How it works…
There's more…
See also
Integrating Elasticsearch with Kafka
Getting ready
How to do it...
How it works…
There's more…
See also
Integrating SolrCloud with Kafka
Getting ready
How to do it...
How it works…
See also
8. Monitoring Kafka
Introduction
Monitoring server stats
Getting ready
How to do it…
How it works…
See also
Monitoring producer stats
Getting ready
How to do it...
How it works…
See also
Monitoring consumer stats
Getting ready
How to do it...
How it works…
See also
Connecting to Graphite
Getting ready
How to do it...
How it works…
See also
Monitoring with Ganglia
Getting ready
How to do it...
How it works…
See also
Index