Thursday, February 18, 2016

Setup 2 node Apache Kafka cluster on Mac-OSX

This is a quick start guide to setup 2 node (broker) cluster on the Mac-OSX.
Steps to install:
1. Download apache kafka from the
2. extract to some folder in my example i have created in /tmp folder
3. Create kafka logs directory
mkdir /tmp/kafka-logs-1
mkdir /tmp/kafka-logs-2
4. copy config/ to config/ as I will be running the two broker on the same machine so the following property needs to be updated in
edit the accordingly
5. now start the zookeeper with
bin/ config/
6. Start kafka broker
bin/ config/ &

7. Start second kafka broker
bin/ config/ &

8. create kafka topic
bin/ --zookeeper localhost:2181 --create --topic general_topic --partitions 2 --replication-factor 2

9. list topic
bin/ --zookeeper localhost:2181 --describe --topic general_topic

10. now its time to test kafka setup, for that setup producer and consumers
bin/ --broker-list  localhost:9092 --topic general_topic

bin/ --zookeeper localhost:2181 --topic general_topic
bin/ --zookeeper localhost:2181 --topic general_topic --from-beginning

Post a Comment