Hi,
In our cluster (across many), rarely I see a deleted key reappearing during consume. This happens in kafka 3.2 and also in 3.9 (2 versions used in various cluster)
Example:
Key:abc (offset 100) - sent at T1
Tombstone for key:abc - sent at T1 + 2 days
(Tombstone delete.retention.ms is 1 day)
Consume at T1 + 4 days (rehydration - consume from beginning) - No key
Consume at T1 + 6 days (rehydration - consume from beginning) - key:abc appears with same offset(100)
These number of days varies but I get this issue approximately 1% of time (not consistent)
Anybody faced this issue and any solution recommended. My topic config is below
Partition=10
Replication-factor=5
min.insync.replicas=2
cleanup.policy=compact
segment.bytes=15728640
min.compaction.lag.ms=900000
max.compaction.lag.ms=1800000
delete.retention.ms=86400000
retention.ms=259200000
message.timestamp.type=CreateTime
Regards,
Nanda
In our cluster (across many), rarely I see a deleted key reappearing during consume. This happens in kafka 3.2 and also in 3.9 (2 versions used in various cluster)
Example:
Key:abc (offset 100) - sent at T1
Tombstone for key:abc - sent at T1 + 2 days
(Tombstone delete.retention.ms is 1 day)
Consume at T1 + 4 days (rehydration - consume from beginning) - No key
Consume at T1 + 6 days (rehydration - consume from beginning) - key:abc appears with same offset(100)
These number of days varies but I get this issue approximately 1% of time (not consistent)
Anybody faced this issue and any solution recommended. My topic config is below
Partition=10
Replication-factor=5
min.insync.replicas=2
cleanup.policy=compact
segment.bytes=15728640
min.compaction.lag.ms=900000
max.compaction.lag.ms=1800000
delete.retention.ms=86400000
retention.ms=259200000
message.timestamp.type=CreateTime
Regards,
Nanda
Comments
Post a Comment