在Linux系統中配置Apache Kafka涉及多個步驟,包括設置服務器屬性、Zookeeper連接以及Kafka broker的配置。以下是一個基本的Kafka配置文件示例和說明:
首先,確保你已經安裝了Kafka。你可以從Kafka官方網站下載并按照安裝指南進行安裝。
Kafka使用Zookeeper來管理集群狀態。你需要先啟動Zookeeper。
zoo.cfg)tickTime=2000
dataDir=/var/lib/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
bin/zookeeper-server-start.sh config/zoo.cfg
Kafka的配置文件是server.properties,通常位于config目錄下。
server.properties)# The directory under which the log data will be stored.
log.dirs=/tmp/kafka-logs
# The number of partitions for the default topic.
num.partitions=1
# The default number of log partitions per broker.
default.replication.factor=1
# The expected number of connections to the broker.
connections.max.idle.ms=60000
# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=localhost:2181
# Timeout in ms for client connection keeping alive checks.
session.timeout.ms=10000
# Timeout in ms used to detect if a producer has stalled
request.timeout.ms=30000
# The maximum size of the request body that the server will accept.
max.request.size=1048576
# The maximum size of the log segment files. When this is reached a new
# log segment will be created.
log.segment.bytes=1073741824
# The interval at which log segments are checked and possibly deleted.
log.retention.hours=168
# The retention period for logs. Logs older than this will be deleted.
log.retention.check.interval.ms=300000
# Zookeeper connection timeout (in ms). This configuration specifies the
# time to wait for a response from Zookeeper before considering the
# connection to be lost.
zookeeper.connection.timeout.ms=18000
bin/kafka-server-start.sh config/server.properties
創建一個Topic來測試Kafka是否正常工作。
bin/kafka-topics.sh --create --topic test-topic --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1
你可以使用以下命令來驗證Kafka是否正常運行:
bin/kafka-topics.sh --list --bootstrap-server localhost:9092
bin/kafka-console-producer.sh --topic test-topic --bootstrap-server localhost:9092
bin/kafka-console-consumer.sh --topic test-topic --from-beginning --bootstrap-server localhost:9092
通過以上步驟,你應該能夠在Linux系統上成功配置和運行Kafka。根據你的具體需求,可能需要調整一些配置參數。