Hi
the description of that blog is correct. as the default value of retries
has been change to max_int from 0, then the order is not guaranteed in
single partition any more by default even if records was send from same
producer.
the following description come from "Kafka: The Definitive Guide"
Apache Kafka preserves the order of messages within a partition. This means
that if messages were sent from the producer in a spe‐ cific order, the
broker will write them to a partition in that order and all consumers will
read them in that order. For some use cases, order is very important. There
is a big difference between deposit‐ ing $100 in an account and later
withdrawing it, and the other way around! However, some use cases are less
sensitive.
Setting the retries parameter to nonzero and the
max.in.flights.requests.per.session
to more than one means that it is possible that the broker will fail to
write the first batch of messages, succeed to write the second (which was
already in- flight), and then retry the first batch and succeed, thereby
reversing the order.
Usually, setting the number of retries to zero is not an option in a
reliable system, so if guaranteeing order is critical, we recommend
setting in.flight.requests.per.session=1
to make sure that while a batch of messages is retrying, additional
messages will not be sent (because this has the potential to reverse the
correct order). This will severely limit the throughput of the producer, so
only use this when order is important.
Best,
Lisheng
Jose Manuel Vega Monroy <jose.monroy@williamhill.com> 于2019年8月29日周四
下午10:11写道：
> Hi there,
>
>
>
> Recently we found this blog entry about *a possible problem with messages
> order:*
> https://blog.softwaremill.com/does-kafka-really-guarantee-the-order-of-messages-3ca849fd19d2
>
>
>
> Please could you confirm about? And if so, how to fix? Blog entry
> suggesting how to.
>
>
>
> Thanks
>
>
>
> [image:
> https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]
> <http://www.williamhill.com/>
>
> [image:
> https://www.williamhillplc.com/content/signature/senet.gif?width=180]
> <http://www.whenthefunstops.co.uk/>
>
> *Jose Manuel Vega Monroy *
> *Java Developer / Software Developer Engineer in Test*
>
> Direct: +*0035 0 2008038 (Ext. 8038)*
> Email: jose.monroy@williamhill.com
>
> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>
>
>
>
>
>
>
the description of that blog is correct. as the default value of retries
has been change to max_int from 0, then the order is not guaranteed in
single partition any more by default even if records was send from same
producer.
the following description come from "Kafka: The Definitive Guide"
Apache Kafka preserves the order of messages within a partition. This means
that if messages were sent from the producer in a spe‐ cific order, the
broker will write them to a partition in that order and all consumers will
read them in that order. For some use cases, order is very important. There
is a big difference between deposit‐ ing $100 in an account and later
withdrawing it, and the other way around! However, some use cases are less
sensitive.
Setting the retries parameter to nonzero and the
max.in.flights.requests.per.session
to more than one means that it is possible that the broker will fail to
write the first batch of messages, succeed to write the second (which was
already in- flight), and then retry the first batch and succeed, thereby
reversing the order.
Usually, setting the number of retries to zero is not an option in a
reliable system, so if guaranteeing order is critical, we recommend
setting in.flight.requests.per.session=1
to make sure that while a batch of messages is retrying, additional
messages will not be sent (because this has the potential to reverse the
correct order). This will severely limit the throughput of the producer, so
only use this when order is important.
Best,
Lisheng
Jose Manuel Vega Monroy <jose.monroy@williamhill.com> 于2019年8月29日周四
下午10:11写道：
> Hi there,
>
>
>
> Recently we found this blog entry about *a possible problem with messages
> order:*
> https://blog.softwaremill.com/does-kafka-really-guarantee-the-order-of-messages-3ca849fd19d2
>
>
>
> Please could you confirm about? And if so, how to fix? Blog entry
> suggesting how to.
>
>
>
> Thanks
>
>
>
> [image:
> https://www.williamhillplc.com/content/signature/WHlogo.gif?width=180]
> <http://www.williamhill.com/>
>
> [image:
> https://www.williamhillplc.com/content/signature/senet.gif?width=180]
> <http://www.whenthefunstops.co.uk/>
>
> *Jose Manuel Vega Monroy *
> *Java Developer / Software Developer Engineer in Test*
>
> Direct: +*0035 0 2008038 (Ext. 8038)*
> Email: jose.monroy@williamhill.com
>
> William Hill | 6/1 Waterport Place | Gibraltar | GX11 1AA
>
>
>
>
>
>
>
Comments
Post a Comment