Hi,
I'm using Kafka 2.8
I have a topic with an original retention of 1 day with a large amount of
messages.
We had a problem with some messages being corrupted and we decided to
skip about 3.5 hours of messages; but not all of them are corrupted. So, we
would like to process this period of time again with a modified version of
our consumer application.
In order to avoid the messages being deleted, we extended the retention to
3 days.
But we are now again close to have the messages being deleted and we
cannot extend the retention anymore (we would go over the limit of the disk
space).
So, I would like to extract the messages between the offsets that cover
the 3.5h
Is there a quick way to extract these messages and copy them into another
topic ?
(After that, I want to set the retention back to the original 1 day)
Would kcat help?
I'm using Kafka 2.8
I have a topic with an original retention of 1 day with a large amount of
messages.
We had a problem with some messages being corrupted and we decided to
skip about 3.5 hours of messages; but not all of them are corrupted. So, we
would like to process this period of time again with a modified version of
our consumer application.
In order to avoid the messages being deleted, we extended the retention to
3 days.
But we are now again close to have the messages being deleted and we
cannot extend the retention anymore (we would go over the limit of the disk
space).
So, I would like to extract the messages between the offsets that cover
the 3.5h
Is there a quick way to extract these messages and copy them into another
topic ?
(After that, I want to set the retention back to the original 1 day)
Would kcat help?
Comments
Post a Comment