Hi Daniel,
Kafka 4.0 includes a rewrite of the group coordinator with a new batching mechanism. The batching interval defaults to 5 ms and is controlled by the `group.coordinator.append.linger.ms` broker config option. Could you try lowering this config and rerunning the benchmarks?
Thanks,
Sean
On 2025/08/21 15:50:58 Daniel Germon wrote:
> Hi Greg,
>
> I have changed the linger.ms configuration on the producer and yes it explains the difference in client version latencies, but I am still seeing the broker latency differences with this configuration.
>
> Kafka broker 3.9.1, client 4.0.0, linger.ms = 0
>
> 50th percentile: 310us
> 90th percentile: 414us
> 99th percentile: 825us
> 99.9th percentile: 9813us
>
> Kafka broker 4.0.0, client 4.0.0, linger.ms = 0
>
> 50th percentile: 8193us
> 90th percentile: 10373us
> 99th percentile: 11109us
> 99.9th percentile: 13812us
>
>
> Regards,
> Dan
>
> This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.
>
> LMAX Group is the holding company of LMAX Exchange, LMAX Global and LMAX Digital. Our registered address is Yellow Building, 1A Nicholas Road, London W11 4AN.
Kafka 4.0 includes a rewrite of the group coordinator with a new batching mechanism. The batching interval defaults to 5 ms and is controlled by the `group.coordinator.append.linger.ms` broker config option. Could you try lowering this config and rerunning the benchmarks?
Thanks,
Sean
On 2025/08/21 15:50:58 Daniel Germon wrote:
> Hi Greg,
>
> I have changed the linger.ms configuration on the producer and yes it explains the difference in client version latencies, but I am still seeing the broker latency differences with this configuration.
>
> Kafka broker 3.9.1, client 4.0.0, linger.ms = 0
>
> 50th percentile: 310us
> 90th percentile: 414us
> 99th percentile: 825us
> 99.9th percentile: 9813us
>
> Kafka broker 4.0.0, client 4.0.0, linger.ms = 0
>
> 50th percentile: 8193us
> 90th percentile: 10373us
> 99th percentile: 11109us
> 99.9th percentile: 13812us
>
>
> Regards,
> Dan
>
> This message and its attachments are confidential, may not be disclosed or used by any person other than the addressee and are intended only for the named recipient(s). If you are not the intended recipient, please notify the sender immediately and delete any copies of this message.
>
> LMAX Group is the holding company of LMAX Exchange, LMAX Global and LMAX Digital. Our registered address is Yellow Building, 1A Nicholas Road, London W11 4AN.
Comments
Post a Comment