Kafka

Hi, Following up with the test results: Unit/integration tests: https://github.com/apache/kafka/actions/runs/25053481746 System tests: Core: https://drive.google.com/file/d/1k7Vblxai4n1hV6mPlJ4O_Zp5zJx3OYuC/view?usp=sharing 1 failure: test_rate_device_name=eth0_latency_ms=50_metadata_quorum=COMBINED_KRAFT_rate_limit_kbit=1000000_task_name=rate-1000-latency-50 kafkatest.tests.core.network_degrade_test The test consistently fails in my environment Non Core: https://drive.google.com/file/d/16Fg7IcKOl5l3sbfcLjD08Wd6yWEP2ofx/view?usp=sharing 1 failure: test_replica_lags_metadata_quorum=ISOLATED_KRAFT kafkatest.tests.tools.replica_verification_test The test is flaky, it passed in a rerun. Thanks, Mickael On Tue, Apr 28, 2026 at 4:57 PM Mickael Maison <mickael.maison@gmail.com> wrote: > > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of Apache Kafka 4.3.0. > > This release includes ...

Congratulations Sean. On Thu, 30 Apr, 2026, 00:18 Chia-Ping Tsai, <chia7712@gmail.com> wrote: > The Project Management Committee (PMC) for Apache Kafka is pleased to > announce Sean Quah as a new Kafka committer. > > Sean is a true Kafka expert who has made numerous significant contributions > to the project. If you are amazed by the new consumer protocol, we owe a > huge thanks to Sean for helping build this foundational feature. Honestly, > I am so proud to be the one sharing this exciting news with you all. > > Please join me in welcoming and congratulating Sean! > > Chia-Ping Tsai > On behalf of the Apache Kafka PMC >

Congrats! On 4/29/26 12:39 PM, Lianet Magrans wrote: > Congratulations Sean!!! > > On Wed, Apr 29, 2026, 3:19 p.m. Mickael Maison <mickael.maison@gmail.com> > wrote: > >> Congratulations Sean! >> >> On Wed, Apr 29, 2026 at 9:05 PM Dongnuo Lyu via dev >> <dev@kafka.apache.org> wrote: >>> >>> Congrats Sean!! >> >

Congratulations Sean! On Wed, Apr 29, 2026 at 2:49 PM Chia-Ping Tsai <chia7712@gmail.com> wrote: > The Project Management Committee (PMC) for Apache Kafka is pleased to > announce Sean Quah as a new Kafka committer. > > Sean is a true Kafka expert who has made numerous significant contributions > to the project. If you are amazed by the new consumer protocol, we owe a > huge thanks to Sean for helping build this foundational feature. Honestly, > I am so proud to be the one sharing this exciting news with you all. > > Please join me in welcoming and congratulating Sean! > > Chia-Ping Tsai > On behalf of the Apache Kafka PMC >

Hello team, I have a question, your help would be greatly appreciated. KRaft: enabling ACLs + OAuth on an existing cluster required a full reformat, is this expected? Apache Kafka 3.9, KRaft mode, 3 nodes (combined controller+broker). Cluster originally formatted and running with no authorizer and PLAINTEXT only. Goal: add StandardAuthorizer + SASL_SSL/OAUTHBEARER without data loss. What I observed: after updating server.properties on all nodes (authorizer class, super.users, OAuth listener config) and rolling restart, the brokers came up but logs showed what looked like a state mismatch, controllers/brokers behaving as if part of them were still on the pre-change config (old listener names, missing principal context for inter-broker traffic). What worked: stop all nodes, wipe log.dirs and the metadata log, kafka-storage.sh format with the authorizer + OAuth config already in server.properties, start fresh. Clean cluster, ACLs and OAuth working immediately. Qu...

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 4.3.0. This release includes 25 KIPs bringing improvements to all components. Note that we still have 1 blocker (KAFKA-20441) so there will be another RC. Release notes for the 4.3.0 release: https://dist.apache.org/repos/dist/dev/kafka/4.3.0-rc0/RELEASE_NOTES.html Please download, test and vote by May 4, 5pm CEST Kafka's KEYS file containing PGP keys we use to sign the release: https://downloads.apache.org/kafka/KEYS * Release artifacts to be voted upon (source and binary): https://dist.apache.org/repos/dist/dev/kafka/4.3.0-rc0/ * Docker release artifacts to be voted upon: apache/kafka:4.3.0-rc0 apache/kafka-native:4.3.0-rc0 * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://dist.apache.org/repos/dist/dev/kafka/4.3.0-rc0/javadoc/index.html * Tag to be voted upo...

Hi Lucas, This was indeed the issue, and your fix resolved it. Thanks for the assistance — greatly appreciated. Best, Yves. C2 - Restricted use ________________________________ Van: Lucas Brutschy via users <users@kafka.apache.org> Verzonden: maandag 27 april 2026 13:46 Aan: users@kafka.apache.org <users@kafka.apache.org> CC: Lucas Brutschy <lbrutschy@confluent.io> Onderwerp: Re: Kafka Streams 4.1.2 offsets not being committed after upgrade from Spring Boot 3.5.8 to 4.0.5 Hi Yves, do you use a consumer interceptor or wrapper that constructs ConsumerRecord objects using a deprecated pre-4.0 constructor? We have seen these problems when people use these kinds of outdated overloads; see https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FKAFKA%2FKIP-1094%253A%2BAdd%2Ba%2Bnew%2Bconstructor%2Bmethod%2Bwith%2BnextOffsets%2Bto%2BConsumerRecords&data=05%7C02%7Cyves.stroucken%40soprasteria.co...

Hi Yves, do you use a consumer interceptor or wrapper that constructs ConsumerRecord objects using a deprecated pre-4.0 constructor? We have seen these problems when people use these kinds of outdated overloads; see https://cwiki.apache.org/confluence/display/KAFKA/KIP-1094%3A+Add+a+new+constructor+method+with+nextOffsets+to+ConsumerRecords. If that is the case, the solution is to use the new overload. Cheers, Lucas On Fri, Apr 24, 2026 at 5:58 PM STROUCKEN Yves <yves.stroucken@soprasteria.com> wrote: > > Hi Bill, > > Thanks for the quick reply. > Here is some of the output I get with debug logging and kafka-consumer-groups.sh > > ./kafka-consumer-groups.sh \ > --bootstrap-server localhost:9092 \ > --describe \ > --group 'MyProcessingApplication' > > loging.level.org.apache.kafka.streams.processor.internals.StreamTask: DEBUG > > Starting with a clean topic with no events. > > output: ...

Hi Bill, Thanks for the quick reply. Here is some of the output I get with debug logging and kafka-consumer-groups.sh ./kafka-consumer-groups.sh \ --bootstrap-server localhost:9092 \ --describe \ --group 'MyProcessingApplication' loging.level.org.apache.kafka.streams.processor.internals.StreamTask: DEBUG Starting with a clean topic with no events. output: GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID MyProcessingApplication myTopic.0 4 - 0 - MyProcessingApplication-509fdb18-f2ec-43f9-a214-50c3d309fde5-StreamThread-1-consumer-6a48934d-d93f-41d5-a89b-4faecaaba720 / hostIp MyProcessingApplication-509fdb18-f2ec-43f9-a214-50c3d309fde5-StreamThread-1-consumer MyProcessingApplication myTo...

Sorry to hear about the issues. Can you provide log files to help diagnose the problem? Thanks, Bill On Fri, Apr 24, 2026 at 4:53 AM STROUCKEN Yves < yves.stroucken@soprasteria.com> wrote: > Hello Kafka users, > I am looking for help with a Kafka Streams offset commit issue that > started after upgrading. > We have a Spring Cloud Stream application using the Kafka Streams binder. > Old working stack: > > * Spring Boot 3.5.9 > * Spring Cloud 2025.0.0 > * spring-cloud-stream-binder-kafka-streams 4.3.0 > * kafka-streams 3.9.1 > New stack with the issue: > > * Spring Boot 4.0.5 > * Spring Cloud 2025.1.1 > * spring-cloud-stream-binder-kafka-streams 5.0.1 > * kafka-streams 4.1.2 > Broker version: > > * Kafka brokers 3.9.0 > Symptoms: > > * The Kafka Streams application processes new records normally. > * However, committed offsets for the S...

Hello Kafka users, I am looking for help with a Kafka Streams offset commit issue that started after upgrading. We have a Spring Cloud Stream application using the Kafka Streams binder. Old working stack: * Spring Boot 3.5.9 * Spring Cloud 2025.0.0 * spring-cloud-stream-binder-kafka-streams 4.3.0 * kafka-streams 3.9.1 New stack with the issue: * Spring Boot 4.0.5 * Spring Cloud 2025.1.1 * spring-cloud-stream-binder-kafka-streams 5.0.1 * kafka-streams 4.1.2 Broker version: * Kafka brokers 3.9.0 Symptoms: * The Kafka Streams application processes new records normally. * However, committed offsets for the Streams application do not appear to advance. * In Kafka UI, consumer group lag stays high for all partitions, while application-level metrics show near real-time processing of new events. * On restart, the application starts reading from the beginning of the topic again. * When I set auto.offset.reset=no...

Hi Everyone, I wanted to understand if there is a way to recover topic data from Kafka tiered storage after the kafka cluster is deleted . The local data is gone but the data in tiered storage still exists. Is there a way to bring up a new kafka cluster and assign the same tiered storage bucket to it so its able to read data from it by resyncing the metadata? Thanks, Vruttant Mankad

Hi Kafka experts, I am running kafka in windows platform. I have controllers (around 5) and brokers running in separate machines During disaster recovery (Say all controller machines data wiped out, or the controller logs folder is corrupt), but broker machines are intact. what is the best practice to bring the kraft controller back with metadata info? Regards, Nanda

Hello, forwarding a message from the ASF Travel Assistance Committee: > The Travel Assistance Committee are now accepting applications for Community Over Code Asia, 2026. To be held in Beijing, China between the 7th and the 9th August. The Application deadline is set at the 29th of May - so plenty of time to apply. If you require a Visa then please do apply early, do not wait until you know if you are accepted or not. Same for Speakers, you should apply even though you do not know if you will be accepted, so that your application is in. Places are limited for Travel Assistance, so those deemed most in need will get higher priority. If you do not need assistance to get to Beijing, but you know of others who might be interested, then feel free to spread the word! > > Good luck to all those that apply. For more details, checkout https://tac.apache.org/ -Matthias

Hi, while testing Kafka 3.9 (KRaft), we encountered Unclean Leader Elections (ULE) during a partition reassignment. We found an existing issue that describes the problem: https://issues.apache.org/jira/browse/KAFKA-19148 (https://issues.apache.org/jira/browse/KAFKA-19148) As suggested in the issue, the ULE appears to be a false positive and is fixed in Kafka 4.1 (which we have verified). However, when reproducing the problem using the README attached to KAFKA- 19148, we observed message loss (not only reordering) while producing to the cluster with acks=1 (--producer-property acks=1) during the reassignment. My question: Is this message loss expected behavior when producing with acks=1 during partition reassignment? Notes: - The KAFKA-19148 issue has not received any updates for a long time (and is primarily about ULE). - I understand that acks=1 increases throughput but is not safe if the leader is removed imme...

Severity: moderate Affected versions: - Apache Kafka 0.11.0 through 3.9.1 - Apache Kafka 4.0.0 - Apache Kafka Clients (org.apache.kafka:kafka-clients) 0.11.0 through 3.9.1 - Apache Kafka Clients (org.apache.kafka:kafka-clients) 4.0.0 Description: Information exposure vulnerability has been identified in Apache Kafka. The NetworkClient component will output entire requests and responses information in the DEBUG log level in the logs. By default, the log level is set to INFO level. If the DEBUG level is enabled, the sensitive information will be exposed via the requests and responses output log. The entire lists of impacted requests and responses are: * AlterConfigsRequest * AlterUserScramCredentialsRequest * ExpireDelegationTokenRequest * IncrementalAlterConfigsRequest * RenewDelegationTokenRequest * SaslAuthenticateRequest * createDelegationTokenResponse * describeDelegationTokenResponse * SaslAuthenticateRespon...

Severity: important Affected versions: - Apache Kafka 4.1.0 through 4.1.1 Description: A possible security vulnerability has been identified in Apache Kafka. By default, the broker property `sasl.oauthbearer.jwt.validator.class` is set to `org.apache.kafka.common.security.oauthbearer.DefaultJwtValidator`. It accepts any JWT token without validating its signature, issuer, or audience. An attacker can generate a JWT token from any issuer with the `preferred_username` set to any user, and the broker will accept it. We advise the Kafka users using kafka v4.1.0 or v4.1.1 to set the config `sasl.oauthbearer.jwt.validator.class` to `org.apache.kafka.common.security.oauthbearer.BrokerJwtValidator` explicitly to avoid this vulnerability. Since Kafka v4.1.2 and v4.2.0 and later, the issue is fixed and will correctly validate the JWT token. Credit: Павел Романов <promanov1994@gmail.com> (finder) References: https://kafka.apache.org/cve-list https://kafka.apac...

Jun, thanks for your interest. We follow self-serve model. I just granted you permission to assign tickets. Side note: if you contribute, you might want to subscribe to the dev mailing list, which is more suitable for this purpose. -Matthias On 4/10/26 6:45 PM, 조형준 wrote: > Hi, > > I'd like to contribute to Apache Kafka. Could you please add me to the > Kafka contributors group on JIRA? I'm interested in working on KAFKA-20364 > > JIRA ID: xxxxxxjun > > Thanks, Jun >

Hello Kafka Community, We have a use case that requires attaching custom metadata to topics — things like SLA, durability guarantees, and PagerDuty account references. While exploring this in Kafka 4.2.0, we noticed an interesting inconsistency between broker-level and topic-level configuration behavior. Broker-level custom configs work fine: Running the following command succeeds without errors: ./bin/kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type brokers --entity-name 1 \ --alter --add-config 'durability.level=1' And describing the broker returns the config as expected: ./bin/kafka-configs.sh --bootstrap-server localhost:9092 \ --entity-type brokers --entity-name 1 \ --describe Output: Dynamic configs for broker 1 are: durability.level=null sensitive=true synonyms={DYNAMIC_BROKER_CONFIG:durability.level=null} Topic-level custom configs throw an exception: Running a similar command on a topic: ./bin/kafka-...

Hi Manikumar, The CVE seems to be still reserved and not published yet. Best, On Tue, Apr 7, 2026 at 1:49 PM Manikumar <manikumar@apache.org> wrote: > Severity: moderate > > Affected versions: > > - Apache Kafka Clients (org.apache.kafka:kafka-clients) 2.8.0 through 3.9.1 > - Apache Kafka Clients (org.apache.kafka:kafka-clients) 4.0.0 through 4.0.1 > - Apache Kafka Clients (org.apache.kafka:kafka-clients) 4.1.0 through 4.1.1 > > Description: > > A race condition in the Apache Kafka Java producer client’s buffer > pool management can cause messages to be silently delivered to > incorrect topics. > > When a produce batch expires due to delivery.timeout.ms while a > network request containing that batch is still in flight, the batch’s > ByteBuffer is prematurely deallocated and returned to the buffer pool. > If a subsequent producer batch—potentially destined for a different > topic—reuses this fr...

Severity: moderate Affected versions: - Apache Kafka Clients (org.apache.kafka:kafka-clients) 2.8.0 through 3.9.1 - Apache Kafka Clients (org.apache.kafka:kafka-clients) 4.0.0 through 4.0.1 - Apache Kafka Clients (org.apache.kafka:kafka-clients) 4.1.0 through 4.1.1 Description: A race condition in the Apache Kafka Java producer client’s buffer pool management can cause messages to be silently delivered to incorrect topics. When a produce batch expires due to delivery.timeout.ms while a network request containing that batch is still in flight, the batch’s ByteBuffer is prematurely deallocated and returned to the buffer pool. If a subsequent producer batch—potentially destined for a different topic—reuses this freed buffer before the original network request completes, the buffer contents may become corrupted. This can result in messages being delivered to unintended topics without any error being reported to the producer. Data Confidentiality: Messages in...