Hi.
We've run into a situation where Kafka cluster was unstable, but some brokers were still up and responding.
Some of the consumers restarted at that time and were not able to get their commit offset.
We run with auto.offset.reset earliest by default, for bootstrap; after some time, these consumers reset their commit offset to earliest and started reprocessing a bunch of events.
We are using Confluent.Kafka client.
Is that an expected behavior?
Is there an option to only reset offset on the positive ack that the offset is not stored for this consumer?
We'd like the cases when the offset cannot be retrieved due to a transient condition to result in retries, or at least a failure.
We've run into a situation where Kafka cluster was unstable, but some brokers were still up and responding.
Some of the consumers restarted at that time and were not able to get their commit offset.
We run with auto.offset.reset earliest by default, for bootstrap; after some time, these consumers reset their commit offset to earliest and started reprocessing a bunch of events.
We are using Confluent.Kafka client.
Is that an expected behavior?
Is there an option to only reset offset on the positive ack that the offset is not stored for this consumer?
We'd like the cases when the offset cannot be retrieved due to a transient condition to result in retries, or at least a failure.
Comments
Post a Comment