Ok. Thanks for the clarification.
On Thu, Feb 19, 2026 at 8:04 PM Chia-Ping Tsai <chia7712@gmail.com> wrote:
> Hi Subra,
>
> It seems to me both use an async approach. The root cause is that the
> returned error code is different. Please check
> https://issues.apache.org/jira/browse/KAFKA-20089 for more details.
>
> Thanks,
> Chia-Ping
>
> Subra I <iamsubra100@gmail.com> 於 2026年2月19日週四 下午9:58寫道：
>
> > Thanks for the response Lianet.
> >
> > This problem happens only in kraft mode brokers. Works fine if we use
> kafka
> > broker in non-kraft mode. (WIth zookeeper)
> >
> > From what I understand: (I may be wrong. this is what I read online)
> >
> > In zookeeper mode, on getting a partitionsFor request, the broker
> creates a
> > topic if not present and then immediately returns the number of
> partitions.
> > All this happens synchronously.
> >
> > In kraft mode, the broker creates a topic asynchronously. So, unless u
> > retry, you wont get the total number of partitions.
> >
> > Like I said, this is what I read.
> >
> > On Tue, Feb 17, 2026 at 3:19 AM Lianet Magrans <lianetm@apache.org>
> wrote:
> >
> > > Hi Anand,
> > >
> > > The partitionsFor API does not guarantee that a single call will return
> > the
> > > data if the topic does not exist. It issues a first metadata request to
> > > create the topic, and waits for the api timeout for a response (timeout
> > > param or default.api.timeout.ms config). Then returns whatever it gets
> > > from
> > > the broker in that single call (empty if the topic is not found within
> > the
> > > timeout, described in the java docs
> > >
> > >
> >
> https://github.com/apache/kafka/blob/e678b4bb7ca99c7d1be0d554ffe7e66f584771d6/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L1415
> > > ).
> > > Also there are old integration tests with the pattern you describe: a
> > first
> > > call to ensure topic is created, then a following one expected to
> > retrieve
> > > the partitions for the topic (e.g, in 3.9 branch
> > >
> > >
> >
> https://github.com/apache/kafka/blob/5e9866f43ab8e7e41ef39e5584ac50019381328d/core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala#L176
> > > )
> > >
> > > That being said, I wonder if the difference you're mentioning is due to
> > > running with Zookeeper in 3.9 vs 4.x with Kraft? Was that the case?
> > > Not sure if tuning some Kraft configs on your cluster could help (I'm
> not
> > > familiar with the kraft details really), but metadata is eventually
> > > consistent anyways, so for sure it would be advisable to use the
> > > partitionsFor considering that it may require several calls if the
> topic
> > is
> > > not found. From there, we can still consider improving for sure (e.g,
> > > improve the client to continue attempting to find the topic while there
> > is
> > > time?). If you can file a jira with your setup (broker configs, params
> > > passed to the client API) and behaviour, we can better look into it.
> > >
> > > Thanks!
> > > Lianet
> > >
> > > On Tue, Jan 20, 2026 at 8:03 AM Subra I <iamsubra100@gmail.com> wrote:
> > >
> > > > Hello All,
> > > >
> > > > I am using Kafka-clients-4.1.1.jar with a kafka broker that is on
> > version
> > > > 4.0.0. From my Java code, when I call the KafkaConsumer partitionsFor
> > > API,
> > > > I see that it works when the topic is already created in kafka. But
> if
> > it
> > > > is not yet created, the above API call returns empty - no
> information.
> > > >
> > > > I noticed that just before I call partitionsFor API, the topic is not
> > > > created. But after I call this API, the topic gets created, but this
> > API
> > > > still returns no information. Of course, if I call the same API
> again,
> > I
> > > do
> > > > get valid information for this topic.
> > > >
> > > > I used to have kafka-clients-3.9.0.jar earlier. There both these
> cases
> > > were
> > > > working fine. Is this a new change in behavior in
> > > kafka-clients-4.1.1.jar?
> > > >
> > > > Thanks,
> > > > Anand
> > > >
> > >
> >
>
On Thu, Feb 19, 2026 at 8:04 PM Chia-Ping Tsai <chia7712@gmail.com> wrote:
> Hi Subra,
>
> It seems to me both use an async approach. The root cause is that the
> returned error code is different. Please check
> https://issues.apache.org/jira/browse/KAFKA-20089 for more details.
>
> Thanks,
> Chia-Ping
>
> Subra I <iamsubra100@gmail.com> 於 2026年2月19日週四 下午9:58寫道：
>
> > Thanks for the response Lianet.
> >
> > This problem happens only in kraft mode brokers. Works fine if we use
> kafka
> > broker in non-kraft mode. (WIth zookeeper)
> >
> > From what I understand: (I may be wrong. this is what I read online)
> >
> > In zookeeper mode, on getting a partitionsFor request, the broker
> creates a
> > topic if not present and then immediately returns the number of
> partitions.
> > All this happens synchronously.
> >
> > In kraft mode, the broker creates a topic asynchronously. So, unless u
> > retry, you wont get the total number of partitions.
> >
> > Like I said, this is what I read.
> >
> > On Tue, Feb 17, 2026 at 3:19 AM Lianet Magrans <lianetm@apache.org>
> wrote:
> >
> > > Hi Anand,
> > >
> > > The partitionsFor API does not guarantee that a single call will return
> > the
> > > data if the topic does not exist. It issues a first metadata request to
> > > create the topic, and waits for the api timeout for a response (timeout
> > > param or default.api.timeout.ms config). Then returns whatever it gets
> > > from
> > > the broker in that single call (empty if the topic is not found within
> > the
> > > timeout, described in the java docs
> > >
> > >
> >
> https://github.com/apache/kafka/blob/e678b4bb7ca99c7d1be0d554ffe7e66f584771d6/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L1415
> > > ).
> > > Also there are old integration tests with the pattern you describe: a
> > first
> > > call to ensure topic is created, then a following one expected to
> > retrieve
> > > the partitions for the topic (e.g, in 3.9 branch
> > >
> > >
> >
> https://github.com/apache/kafka/blob/5e9866f43ab8e7e41ef39e5584ac50019381328d/core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala#L176
> > > )
> > >
> > > That being said, I wonder if the difference you're mentioning is due to
> > > running with Zookeeper in 3.9 vs 4.x with Kraft? Was that the case?
> > > Not sure if tuning some Kraft configs on your cluster could help (I'm
> not
> > > familiar with the kraft details really), but metadata is eventually
> > > consistent anyways, so for sure it would be advisable to use the
> > > partitionsFor considering that it may require several calls if the
> topic
> > is
> > > not found. From there, we can still consider improving for sure (e.g,
> > > improve the client to continue attempting to find the topic while there
> > is
> > > time?). If you can file a jira with your setup (broker configs, params
> > > passed to the client API) and behaviour, we can better look into it.
> > >
> > > Thanks!
> > > Lianet
> > >
> > > On Tue, Jan 20, 2026 at 8:03 AM Subra I <iamsubra100@gmail.com> wrote:
> > >
> > > > Hello All,
> > > >
> > > > I am using Kafka-clients-4.1.1.jar with a kafka broker that is on
> > version
> > > > 4.0.0. From my Java code, when I call the KafkaConsumer partitionsFor
> > > API,
> > > > I see that it works when the topic is already created in kafka. But
> if
> > it
> > > > is not yet created, the above API call returns empty - no
> information.
> > > >
> > > > I noticed that just before I call partitionsFor API, the topic is not
> > > > created. But after I call this API, the topic gets created, but this
> > API
> > > > still returns no information. Of course, if I call the same API
> again,
> > I
> > > do
> > > > get valid information for this topic.
> > > >
> > > > I used to have kafka-clients-3.9.0.jar earlier. There both these
> cases
> > > were
> > > > working fine. Is this a new change in behavior in
> > > kafka-clients-4.1.1.jar?
> > > >
> > > > Thanks,
> > > > Anand
> > > >
> > >
> >
>
Comments
Post a Comment