Hi Anana,
Typically in Kafka, it is useful for Consumers to know about the number of partitions (as the number of consumers must be <= partitions).
So one way for Consumers to find partitions is the KafkaConsumer class using the partitionsFor(topic) method, https://kafka.apache.org/40/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#partitionsFor(java.lang.String)
Regards, Paul Brebner
From: Subra I <iamsubra100@gmail.com>
Date: Wednesday, 11 June 2025 at 12:42 am
To: users@kafka.apache.org <users@kafka.apache.org>, dev@kafka.apache.org <dev@kafka.apache.org>
Subject: KafkaProducer partitionsFor v/s KafkaAdminClient describeTopics
EXTERNAL EMAIL - USE CAUTION when clicking links or attachments
Hi All,
I need to know the number of partitions for a topic before
producing/consuming. Some users may set the number of partitions for a
given topic but some users may just set the number of partitions in
server.properties. It should work for both cases.
Which method is better for a production grade software:
1. KafkaProducer partitionsFor
2. KafkaAdminClient describeTopics: you need to extract the topic you are
interested in and get the partition info from the metadata.
A quick Google search tells me that describeTopics approach is more
comprehensive.
Also, will the use of KafkaProducer partitionsFor work for clustered
environments?
Thanks,
Anand
Typically in Kafka, it is useful for Consumers to know about the number of partitions (as the number of consumers must be <= partitions).
So one way for Consumers to find partitions is the KafkaConsumer class using the partitionsFor(topic) method, https://kafka.apache.org/40/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#partitionsFor(java.lang.String)
Regards, Paul Brebner
From: Subra I <iamsubra100@gmail.com>
Date: Wednesday, 11 June 2025 at 12:42 am
To: users@kafka.apache.org <users@kafka.apache.org>, dev@kafka.apache.org <dev@kafka.apache.org>
Subject: KafkaProducer partitionsFor v/s KafkaAdminClient describeTopics
EXTERNAL EMAIL - USE CAUTION when clicking links or attachments
Hi All,
I need to know the number of partitions for a topic before
producing/consuming. Some users may set the number of partitions for a
given topic but some users may just set the number of partitions in
server.properties. It should work for both cases.
Which method is better for a production grade software:
1. KafkaProducer partitionsFor
2. KafkaAdminClient describeTopics: you need to extract the topic you are
interested in and get the partition info from the metadata.
A quick Google search tells me that describeTopics approach is more
comprehensive.
Also, will the use of KafkaProducer partitionsFor work for clustered
environments?
Thanks,
Anand
Comments
Post a Comment