Kafka

I am using an external application to authorize requests before sending to broker. Before kraft I initialized aclauthroizer and just called configure and then called authorize (the zk watcher inside aclauthorizer took care of loading and updating acls). In case of kraft, how to use this feature? How can I load the acls before authorize? And how to make sure the changed acls reflect. Regards, Nanda

Hi Anana, Typically in Kafka, it is useful for Consumers to know about the number of partitions (as the number of consumers must be <= partitions). So one way for Consumers to find partitions is the KafkaConsumer class using the partitionsFor(topic) method, https://kafka.apache.org/40/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html#partitionsFor(java.lang.String) Regards, Paul Brebner From: Subra I < iamsubra100@gmail.com > Date: Wednesday, 11 June 2025 at 12:42 am To: users@kafka.apache.org < users@kafka.apache.org >, dev@kafka.apache.org < dev@kafka.apache.org > Subject: KafkaProducer partitionsFor v/s KafkaAdminClient describeTopics EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hi All, I need to know the number of partitions for a topic before producing/consuming. Some users may set the number of partitions for a given topic but some users may just set the number of partitions in server.properties...

Hey Vinothini, Hi and welcome to the Kafka community! 👋 Great to hear you're interested in contributing! Here's how you can get started: a. Getting Familiar with Kafka To get hands-on quickly and understand how Kafka works, we recommend starting with the Kafka Quickstart Guide [1] It walks you through setting up a Kafka broker locally, producing and consuming messages — a great way to build intuition. b. Setting Up Kafka for Local Development If you're planning to make changes to the Kafka codebase or test patches locally: - Clone the repo [2] - Follow the Development README [3] to build the project and run a broker. This will help you: - Build Kafka from source - Run unit and integration tests - Start a Kafka broker locally with custom changes c. Exploring Good First Issues You can browse beginner-friendly issues using the newbie label on Jira [4]. We're glad to have you on board — happy learning! [1]: https://kaf...

Hi Nanda, It's great you figured it out. "KIP-1157 < https://cwiki.apache.org/confluence/display/KAFKA/KIP-1157%3A+Enforce+KafkaPrincipalSerde+Implementation+for+KafkaPrincipalBuilder >: Enforce KafkaPrincipalSerde Implementation for KafkaPrincipalBuilder" is proposed to fix this issue. Thank you. Luke On Wed, Jun 11, 2025 at 12:39 AM Nanda Naga <nandanaga@microsoft.com.invalid> wrote: > I figured this out issue - it is due to missing > serialization/deserialization logic for the custom principal > > Regards, > Nanda > > -----Original Message----- > From: Nanda Naga <nandanaga@microsoft.com.INVALID> > Sent: Friday, June 6, 2025 1:19 PM > To: users@kafka.apache.org > Subject: [EXTERNAL] Kraft mode - Authz errors while doing alterconfig via > admin client > > [You don't often get email from nandanaga@microsoft.com.invalid. Learn > why this is important at https://aka.ms/L...

Hi All, Current is a conference dedicated to data streaming, Apache Kafka®, Apache Flink®, and Apache Iceberg™ communities. Now is the time to apply your ideas and experiences to share with the community. The Current 2025 CFP < https://sessionize.com/current-2025-new-orleans/ > is open *until June 15*.

I figured this out issue - it is due to missing serialization/deserialization logic for the custom principal Regards, Nanda -----Original Message----- From: Nanda Naga <nandanaga@microsoft.com.INVALID> Sent: Friday, June 6, 2025 1:19 PM To: users@kafka.apache.org Subject: [EXTERNAL] Kraft mode - Authz errors while doing alterconfig via admin client [You don't often get email from nandanaga@microsoft.com.invalid. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] In broker server properties and controller server properties, I have setup the custom principal builder class name and custom acl authorizer (extends standard authorizer) class name properly The normal produce/ consumes that the topic has acls works fine though using the custom principal and custom acl authorizer. It works when it is inter controller auth calls But when requests sent via admin client (using command prompt calls) or via code that uses admin client, ...

Hi All, I need to know the number of partitions for a topic before producing/consuming. Some users may set the number of partitions for a given topic but some users may just set the number of partitions in server.properties. It should work for both cases. Which method is better for a production grade software: 1. KafkaProducer partitionsFor 2. KafkaAdminClient describeTopics: you need to extract the topic you are interested in and get the partition info from the metadata. A quick Google search tells me that describeTopics approach is more comprehensive. Also, will the use of KafkaProducer partitionsFor work for clustered environments? Thanks, Anand