springboot 整合 kafka 批量處理

在 Spring Boot 中整合 Kafka 進行批量處理，可以通過以下步驟實現：

1. 添加依賴

在 pom.xml 文件中添加 Kafka 和 Spring Boot Kafka 相關的依賴：

<dependencies>
    <!-- Spring Boot Kafka Starter -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-kafka</artifactId>
    </dependency>
</dependencies>

2. 配置 Kafka

在 application.yml 或 application.properties 文件中配置 Kafka 相關參數：

spring:
  kafka:
    bootstrap-servers: localhost:9092
    consumer:
      group-id: my-group
      auto-offset-reset: earliest
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer

3. 創建 Kafka 配置類

創建一個配置類，用于設置 Kafka 生產者和消費者的屬性：

@Configuration
public class KafkaConfig {

    @Bean
    public ProducerFactory<String, String> producerFactory() {
        Map<String, Object> configProps = new HashMap<>();
        configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        return new DefaultKafkaProducerFactory<>(configProps);
    }

    @Bean
    public KafkaTemplate<String, String> kafkaTemplate() {
        return new KafkaTemplate<>(producerFactory());
    }

    @Bean
    public ConsumerFactory<String, String> consumerFactory() {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ConsumerConfig.GROUP_ID_CONFIG, "my-group");
        props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        return new DefaultKafkaConsumerFactory<>(props);
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory() {
        ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }
}

4. 創建 Kafka 監聽器

創建一個監聽器類，用于處理接收到的 Kafka 消息：

@Service
public class KafkaListener {

    @KafkaListener(topics = "my-topic", groupId = "my-group")
    public void listen(List<String> messages) {
        for (String message : messages) {
            System.out.println("Received message: " + message);
            // 在這里進行批量處理
        }
    }
}

5. 發送批量消息

創建一個生產者類，用于向 Kafka 發送批量消息：

@Service
public class KafkaProducer {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendMessages(List<String> messages) {
        kafkaTemplate.send("my-topic", messages);
    }
}

6. 在主應用中調用

在主應用中，可以調用 KafkaProducer 發送批量消息，并監聽這些消息：

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Autowired
    private KafkaProducer kafkaProducer;

    @Autowired
    private KafkaListener kafkaListener;

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);

        Application application = new Application();
        application.kafkaProducer.sendMessages(Arrays.asList("message1", "message2", "message3"));
    }
}

這樣，當發送批量消息到 my-topic 主題時，KafkaListener 將接收到這些消息并進行批量處理。