Christophe,
See "Problems when Consuming from multiple Partitions" in the list archive. I'll forward you the full conversation privately. It includes debug logs that demonstrate fetches being discarded and refetched, but only after one or more full expirations of the fetch.max.wait.ms even though messages are available. It appears to be due to the re-fetch only querying one of the Partitions allocated to the Consumer, so if that Partition is empty you get the full delay even though messages are available (and in fact already fetched) from another Partition.
The exact behaviour differs depending on the server and client versions. I tried 2.2.1 and 2.3.1 servers with a 2.4.0 client. Both introduced delays and/or some partitions not being processed at all.
The problems were only observed where a Consumer subscribed to multiple Partitions of the same Topic.
I haven't raised an issue for it as I would have no expectation of it being fixed. I just made my client and server versions match and implemented a workaround for the issue I wanted the 2.4.0 client to resolve.
Regards, James.
On 31/03/2020, at 12:16, JALADY Christophe <christophe.jalady@laposte.fr<mailto:christophe.jalady@laposte.fr>> wrote:
There are serious latency issues when mixing different client and server version
Could you be more specific ? Link to any issue ?
Thanks by advance !
Christophe
________________________________
De : James Olsen <james@inaseq.com<mailto:james@inaseq.com>>
Envoyé : vendredi 27 mars 2020 01:48
À : users@kafka.apache.org<mailto:users@kafka.apache.org> <users@kafka.apache.org<mailto:users@kafka.apache.org>>
Objet : Re: Reg : Slowness in Kafka
Also check your Kafka Client and Server versions. There are serious latency issues when mixing different client and server versions IF your consumers handle multiple partitions.
On 27/03/2020, at 12:59, Chris Larsen <clarsen@confluent.io<mailto:clarsen@confluent.io>> wrote:
Hi Vidhya,
How many tasks are you running against the topic? How many partitions are
on the topic? Can you post the connector config anonymized?
Best,
Chris
On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar <sakar.black@gmail.com<mailto:sakar.black@gmail.com>> wrote:
Hi Team,
The Kafka consumer is reading only 8 records per second.We have implemented
apache Kafka and confluent connect S3. The confluent connect S3 collects
the records and pushes it to S3 bucket.
In this process, we are seeing some slowness like on an average only 8
records is being processed for a second. am planning to have a higher
throughout results, so that where there is a higher data load, it should
process more number of records.
Thanks in advance.
--
*Chris Larsen*
Systems Engineer | Confluent
847.274.3735
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<https://www.confluent.io/blog/>
Post-scriptum La Poste
Ce message est confidentiel. Sous reserve de tout accord conclu par
ecrit entre vous et La Poste, son contenu ne represente en aucun cas un engagement de la part de La Poste. Toute publication, utilisation ou diffusion, meme partielle, doit etre autorisee prealablement. Si vous n'etes pas destinataire de ce message, merci d'en avertir immediatement
l'expediteur.
See "Problems when Consuming from multiple Partitions" in the list archive. I'll forward you the full conversation privately. It includes debug logs that demonstrate fetches being discarded and refetched, but only after one or more full expirations of the fetch.max.wait.ms even though messages are available. It appears to be due to the re-fetch only querying one of the Partitions allocated to the Consumer, so if that Partition is empty you get the full delay even though messages are available (and in fact already fetched) from another Partition.
The exact behaviour differs depending on the server and client versions. I tried 2.2.1 and 2.3.1 servers with a 2.4.0 client. Both introduced delays and/or some partitions not being processed at all.
The problems were only observed where a Consumer subscribed to multiple Partitions of the same Topic.
I haven't raised an issue for it as I would have no expectation of it being fixed. I just made my client and server versions match and implemented a workaround for the issue I wanted the 2.4.0 client to resolve.
Regards, James.
On 31/03/2020, at 12:16, JALADY Christophe <christophe.jalady@laposte.fr<mailto:christophe.jalady@laposte.fr>> wrote:
There are serious latency issues when mixing different client and server version
Could you be more specific ? Link to any issue ?
Thanks by advance !
Christophe
________________________________
De : James Olsen <james@inaseq.com<mailto:james@inaseq.com>>
Envoyé : vendredi 27 mars 2020 01:48
À : users@kafka.apache.org<mailto:users@kafka.apache.org> <users@kafka.apache.org<mailto:users@kafka.apache.org>>
Objet : Re: Reg : Slowness in Kafka
Also check your Kafka Client and Server versions. There are serious latency issues when mixing different client and server versions IF your consumers handle multiple partitions.
On 27/03/2020, at 12:59, Chris Larsen <clarsen@confluent.io<mailto:clarsen@confluent.io>> wrote:
Hi Vidhya,
How many tasks are you running against the topic? How many partitions are
on the topic? Can you post the connector config anonymized?
Best,
Chris
On Thu, Mar 26, 2020 at 17:58 Vidhya Sakar <sakar.black@gmail.com<mailto:sakar.black@gmail.com>> wrote:
Hi Team,
The Kafka consumer is reading only 8 records per second.We have implemented
apache Kafka and confluent connect S3. The confluent connect S3 collects
the records and pushes it to S3 bucket.
In this process, we are seeing some slowness like on an average only 8
records is being processed for a second. am planning to have a higher
throughout results, so that where there is a higher data load, it should
process more number of records.
Thanks in advance.
--
*Chris Larsen*
Systems Engineer | Confluent
847.274.3735
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<https://www.confluent.io/blog/>
Post-scriptum La Poste
Ce message est confidentiel. Sous reserve de tout accord conclu par
ecrit entre vous et La Poste, son contenu ne represente en aucun cas un engagement de la part de La Poste. Toute publication, utilisation ou diffusion, meme partielle, doit etre autorisee prealablement. Si vous n'etes pas destinataire de ce message, merci d'en avertir immediatement
l'expediteur.
Comments
Post a Comment