k8s kafka怎樣部署

在Kubernetes（k8s）上部署Kafka主要涉及以下幾個步驟：

1. 準備環境

確保你的Kubernetes集群已經正常運行，并且你已經安裝了kubectl和有一個可用的Kubernetes上下文。

2. 創建Kafka命名空間

首先，創建一個命名空間來隔離Kafka資源。

kubectl create namespace kafka

3. 創建Kafka配置文件

創建Kafka的配置文件，包括server.properties和相關的部署文件。以下是一個簡單的示例：

server.properties

broker.id=0
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-logs
zookeeper.connect=zookeeper:2181

kafka-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka
  namespace: kafka
spec:
  replicas: 3
  selector:
    matchLabels:
      app: kafka
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - name: kafka
        image: bitnami/kafka:2.8.1
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_BROKER_ID
          value: "0"
        - name: KAFKA_LISTENERS
          value: "PLAINTEXT://:9092"
        - name: KAFKA_LOG_DIRS
          value: "/tmp/kafka-logs"
        - name: KAFKA_ZOO_CONNECT
          value: "zookeeper:2181"
        volumeMounts:
        - name: kafka-storage
          mountPath: /tmp/kafka-logs
      volumes:
      - name: kafka-storage
        persistentVolumeClaim:
          claimName: kafka-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: kafka-pvc
  namespace: kafka
spec:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 10Gi

4. 創建Zookeeper服務

Kafka依賴于Zookeeper來管理元數據。創建一個Zookeeper服務。

zookeeper-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: zookeeper
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: zookeeper
  template:
    metadata:
      labels:
        app: zookeeper
    spec:
      containers:
      - name: zookeeper
        image: bitnami/zookeeper:3.7.0
        ports:
        - containerPort: 2181
        env:
        - name: ZOO_MY_ID
          value: "1"
        - name: ZOO_SERVERS
          value: "server.1:2888:3888"
      volumes:
      - name: zookeeper-storage
        persistentVolumeClaim:
          claimName: zookeeper-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: zookeeper-pvc
  namespace: kafka
spec:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 10Gi

5. 應用配置文件

使用kubectl應用這些配置文件。

kubectl apply -f kafka-deployment.yaml
kubectl apply -f zookeeper-deployment.yaml

6. 驗證部署

驗證Kafka和Zookeeper是否正常運行。

kubectl get pods -n kafka
kubectl get services -n kafka

7. 訪問Kafka

你可以使用Kafka客戶端工具（如kafka-console-producer.sh和kafka-console-consumer.sh）來驗證Kafka集群是否正常工作。

啟動Kafka生產者

kubectl exec -it <kafka-pod-name> -- /bin/bash
./kafka-console-producer.sh --broker-list <kafka-service-ip>:9092 --topic test

啟動Kafka消費者

kubectl exec -it <kafka-pod-name> -- /bin/bash
./kafka-console-consumer.sh --bootstrap-server <kafka-service-ip>:9092 --topic test --from-beginning

通過以上步驟，你應該能夠在Kubernetes集群上成功部署Kafka和Zookeeper。