Hey there,
I've got the problem that the "__consumer_offsets" topic grows pretty big over time. After some digging, I found offsets for consumer groups that were deleted a long time ago still being present in the topic. Many of them are offsets for console consumers, that have been deleted with "kafka-consumer-groups.sh --delete --group ...".
As far as I understand log cleaning, those offsets should have been deleted a long time ago, because these consumers are no longer active. When I query "kafka-consumer-groups.sh --bootstrap-server ... --list" I don't see those consumers either.
Is there a bug in "kafka-consumer-groups.sh --delete --group ..." that let's kafka hang on to those consumer groups?
How can I get the log cleaner to delete these old offsets? Is there another way than setting "cleanup.policy" to "delete"?
Thanks for our help!
Best,
Claudia
I've got the problem that the "__consumer_offsets" topic grows pretty big over time. After some digging, I found offsets for consumer groups that were deleted a long time ago still being present in the topic. Many of them are offsets for console consumers, that have been deleted with "kafka-consumer-groups.sh --delete --group ...".
As far as I understand log cleaning, those offsets should have been deleted a long time ago, because these consumers are no longer active. When I query "kafka-consumer-groups.sh --bootstrap-server ... --list" I don't see those consumers either.
Is there a bug in "kafka-consumer-groups.sh --delete --group ..." that let's kafka hang on to those consumer groups?
How can I get the log cleaner to delete these old offsets? Is there another way than setting "cleanup.policy" to "delete"?
Thanks for our help!
Best,
Claudia
Comments
Post a Comment