Skip to main content

Offsets of deleted consumer groups do not get deleted correctly

Hey there,

I've got the problem that the "__consumer_offsets" topic grows pretty big over time. After some digging, I found offsets for consumer groups that were deleted a long time ago still being present in the topic. Many of them are offsets for console consumers, that have been deleted with "kafka-consumer-groups.sh --delete --group ...".

As far as I understand log cleaning, those offsets should have been deleted a long time ago, because these consumers are no longer active. When I query "kafka-consumer-groups.sh --bootstrap-server ... --list" I don't see those consumers either.

Is there a bug in "kafka-consumer-groups.sh --delete --group ..." that let's kafka hang on to those consumer groups?

How can I get the log cleaner to delete these old offsets? Is there another way than setting "cleanup.policy" to "delete"?

Thanks for our help!

Best,
Claudia

Comments