Re: sync or async producer

Hello,

I did a test with these two rb scripts which take the time almost the
same. do you have the further idea?

$ cat async-pub.rb
require 'kafka'

kafka = Kafka.new("localhost:9092", client_id: "ruby-client",
resolve_seed_brokers: true)
producer = kafka.async_producer(required_acks: :all,max_buffer_size:
50_000,max_queue_size: 10_000)

10000.times do
message = rand.to_s
producer.produce(message, topic: "mytest")
end

producer.deliver_messages
producer.shutdown


$ cat sync-pub.rb
require 'kafka'

kafka = Kafka.new("localhost:9092", client_id: "ruby-client",
resolve_seed_brokers: true)

producer = kafka.producer(required_acks: :all,max_buffer_size: 50_000)

10000.times do
message = rand.to_s
producer.produce(message, topic: "mytest")
end

producer.deliver_messages

Thanks

On 2022/2/16 10:18, Luke Chen wrote:
> Hi frakass,
>
> I think the most difference for sync and async send (or "publish" like you
> said), is the throughput.
> You said the performance is almost the same, and I would guess the "acks"
> config in your environment might be 0? Or maybe the produce rate is slow?
> Or "max.in.flight.requests.per.connection" is 1?
> Usually, when "acks=all", you have to wait for the records completely
> replicated into all brokers before server response in "sync" mode, which is
> why the throughput will be slow.
> Compared with async mode, the producer send will return immediately after
> appending the records, and wait for the response in callback function, no
> matter it's acks=0 or acks=all.
>
>
> Hope that helps.
>
> Luke
>
>
>
>
>
>
>
> On Wed, Feb 16, 2022 at 9:10 AM frakass <capitnfrakass@free.fr> wrote:
>
>> for a producer, is there a principle that when to use sync publishing,
>> and when to use async publishing?
>>
>> for the simple format messages, i have tested both, their performance
>> are almost the same.
>>
>> Thank you.
>> frakass
>>
>