Skip to main content

delete.retention.ms=0 impact on changelog topics

Hello! We are currently getting hit by https://issues.apache.org/jira/browse/KAFKA-8522. We've tried setting the delete.retention.ms to be very low, but it still doesn't allow our topic to be cleansed of deletes. Setting it to 0 appears to be the only solution.

Our suppress and window changelog topics have min.cleanable.dirty.ratio set to 0.05 (the topic can get pretty big, which makes restores painfully slow), and a segment.ms of 12 hours to allow the topic to stay as small as possible without stressing the log cleaner tooo much.

With this background, my question it: will setting delete.retention.ms to 0 cause any unforeseen side effects for our restoring windows/suppress stores? Would it be possible to consume a valid message and then miss the following tombstone with these settings? And then maybe somehow cause our suppress store to emit an old should-have-already-been-deleted record? Or is the only downside the extra work we are causing in the log cleaner?

Thank you!

Comments