RE: consumer loses offset

Hi, thanks for your response.

Is there any chance the offset is never committed on the "__consumer_offsets" topic, although auto commit is enabled every 5 seconds?
We are checking daily and the offset is always set to NULL.



Lorenzo Rovere

Technology Reply
Via Avogadri, 2
31057 - Silea (TV) - ITALY
phone: +39 0422 1836521
l.rovere@reply.it
www.reply.it
-----Original Message-----
From: Luke Chen <showuon@gmail.com>
Sent: 7 October, 2022 2:15 PM
To: users@kafka.apache.org
Subject: Re: consumer loses offset

Hi Lorenzo,

Sounds like it is caused by this bug:
https://issues.apache.org/jira/browse/KAFKA-13636
If you're not in the versions of fixed version list or newer, please try to upgrade it.

Thanks.
Luke

On Fri, Oct 7, 2022 at 5:36 PM Lorenzo Rovere <l.rovere@reply.it> wrote:

> Hi everyone, I have a simple question about Kafka offsets.
>
> We have 1 producer and 1 consumer.
>
> Imagine the consumer reading till offset 5 (for example) and then
> suddenly stops for some hours. In the meantime the producer keeps
> writing messages and the offset of the last message is 10 (always for
> example). When we restart the microservice that contains the consumer,
> where does it start to read from, if we have auto.offset.reset=latest?
>
>
>
> I ask this because one of our costumers complains about starting
> reading from 10, thus losing messages between 5 and 10. Is that correct?
>
>
>
> Other configs:
> enable.auto.commit = true
>
> auto.commit.interval.ms = 5000
>
> log.retention.hours=168
>
> offsets.retention.minutes (7d)
>
>
>
> We also noticed that on the "__consumer_offsets" topic offset is
> always set to NULL
>
> [consumer_group, topic_name,partition]::NULL
>
>
>
> Can you help me understanding what's happening? Thanks a lot
>
>
> Lorenzo Rovere
>
> Technology Reply
> Via Avogadri, 2
> 31057 - Silea (TV) - ITALY
> phone: +39 0422 1836521
> l.rovere@reply.it
> www.reply.it
>
> [image: Technology Reply]
>