Skip to main content

producer transaction hanging issue

Hi, i have a question about  kafka transaction hanging  issue:

when i restart flink job,  something wrong with kafka transaction.  can't consumer in read_commit status.

1. find-hanging
./bin/kafka-transactions.sh  --command-config ./config/clent_jaas.conf --bootstrap-server 172.18.8.18:9092,172.18.8.19:9092,172.18.8.20:9092 find-hanging --topic ods_csys_20220805_prd_01 --partition 0  
Topic                           Partition       ProducerId      ProducerEpoch   CoordinatorEpoch        StartOffset     LastTimestamp   Duration(min)
ods_csys_20220805_prd_01        0               12241081        0               -1                      13045569        1667275399011   21779        
ods_csys_20220805_prd_01        0               12148922        0               -1                      12863388        1667156500906   23761        
ods_csys_20220805_prd_01        0               12224702        0               -1                      13014865        1667252649837   22158        
ods_csys_20220805_prd_01        0               12175551        0               -1                      12908164        1667192889534   23154        
ods_csys_20220805_prd_01        0               12210364        0               -1                      13001172        1667237047285   22418        
ods_csys_20220805_prd_01        0               12298428        0               -1                      13156486        1667348702381   20557   

2. abort use startoffset

3. kafka serser log 
log:
logs/server.log.2022-08-12-11:[2022-08-12 11:06:16,519] INFO [TransactionCoordinator id=3] Initialized transactionalId ods_csys_realtime_20220805_prd_trx_02-0-9860 with producerId 6385510 and producer epoch 0 on partition __transaction_state-49 (kafka.coordinator.transaction.TransactionCoordinator)
logs/server.log.2022-11-03-08:[2022-11-03 08:24:02,923] INFO [TransactionCoordinator id=3] Initialized transactionalId ods_csys_realtime_20220805_prd_trx_02-0-9860 with producerId 12360851 and producer epoch 0 on partition __transaction_state-49 (kafka.coordinator.transaction.TransactionCoordinator)
logs/server.log.2022-11-10-13:[2022-11-10 13:10:32,774] INFO [TransactionCoordinator id=3] Initialized transactionalId ods_csys_realtime_20220805_prd_trx_02-0-9860 with producerId 12845103 and producer epoch 0 on partition __transaction_state-49 (kafka.coordinator.transaction.TransactionCoordinator)


ods_csys_realtime_20220805_prd_trx_02-0-9860 with producerId  6385510 is normal,     12360851 is hanging ,  12845103 is normal


4. find more infomation
./bin/kafka-console-consumer.sh --bootstrap-server 172.18.8.18:9092 --consumer.config  ./config/clent_jaas.conf  --topic __transaction_state --formatter "kafka.coordinator.transaction.TransactionLog\$TransactionLogMessageFormatter" --from-beginning




i don't know how cause it ?

Comments