Skip to main content

Copying data from one topic into another

Hi,
I'm using Kafka 2.8
I have a topic with an original retention of 1 day with a large amount of
messages.

We had a problem with some messages being corrupted and we decided to
skip about 3.5 hours of messages; but not all of them are corrupted. So, we
would like to process this period of time again with a modified version of
our consumer application.

In order to avoid the messages being deleted, we extended the retention to
3 days.
But we are now again close to have the messages being deleted and we
cannot extend the retention anymore (we would go over the limit of the disk
space).

So, I would like to extract the messages between the offsets that cover
the 3.5h

Is there a quick way to extract these messages and copy them into another
topic ?
(After that, I want to set the retention back to the original 1 day)

Would kcat help?

Comments