Re: Copying data from one topic into another

By the way, we use Avro for the messages.



On Wed, Nov 30, 2022 at 9:35 AM Tecno Brain <cerebrotecnologico@gmail.com>
wrote:

> Hi,
> I'm using Kafka 2.8
> I have a topic with an original retention of 1 day with a large amount
> of messages.
>
> We had a problem with some messages being corrupted and we decided to
> skip about 3.5 hours of messages; but not all of them are corrupted. So, we
> would like to process this period of time again with a modified version of
> our consumer application.
>
> In order to avoid the messages being deleted, we extended the retention
> to 3 days.
> But we are now again close to have the messages being deleted and we
> cannot extend the retention anymore (we would go over the limit of the disk
> space).
>
> So, I would like to extract the messages between the offsets that cover
> the 3.5h
>
> Is there a quick way to extract these messages and copy them into
> another topic ?
> (After that, I want to set the retention back to the original 1 day)
>
> Would kcat help?
>
>
>
>
>