Kafka

Issue https://github.com/dpkp/kafka-python/issues/2676 I've sent request *FetchRequest_v3(replica_id=-1, max_wait_time=500, min_bytes=1, max_bytes=52428800, topics=[(topic='my-secret-topic', partitions=[(partition=0, offset=19857795, max_bytes=1048576), (partition=3, offset=18443170, max_bytes=1048576)])]) * but got a raw value without headers > > *b'\x00\x00\x00F\x00\x00\x00\x01\x00\x00\x00\x01\x00"***x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\xfaOc'* What could be the reason?

Здравствуйте, коллеги. Мы эксплуатируем кластер Kafka 3.6.1 (KRaft mode) с 9 брокерами и 3 отдельными контроллерами. Аппаратные ресурсы брокеров: 24 CPU, 48 GB RAM. Количество соединений: до ~80k одновременных. ------------------------------ *Конфигурация брокеров:* group.initial.rebalance.delay.ms = 0log.retention.check.interval.ms =30000 log.retention.hours=24 log.roll.hours=1 log.segment.bytes=1073741824 num.io.threads=24 num.network.threads=24 num.recovery.threads.per.data.dir=2 offsets.topic.replication.factor=3 socket.receive.buffer.bytes=-1 socket.request.max.bytes=10485760 socket.send.buffer.bytes=-1 transaction.state.log.min.isr=2 transaction.state.log.replication.factor= 3zookeeper.connection.timeout.ms =10000 delete.topic.enable=true replica.fetch.max.bytes=5242880 max.message.bytes=5242880 message.max.bytes=5242880 default.replication.factor=3 min.insync.replicas=2 num.replica.fetchers= 2replica.fetch.wait.max.ms = 500replica.lag.time.max.ms ...

Dear Team, We are looking into running KFKA applications on a consolidated/shared Infrastructure. Please may we have a session from the KAFKA sales team who can take us through on following agenda 1. What is KAFKA 2. Benefits of KAFKA 3. Use cases of KFKA 4. How KAFKA can be deployed 5. Consolidated Architecture for KAFKA/Shares Infrastructure Kind regards, Tumwagile Gedion - Senior Specialist; Dc Servers Infrastructure [ https://nmbsignatures.s3.us-east-2.amazonaws.com/logo.gif ] NMB - Hq Tel: Dar Es Salaam - Tanzania Cell:0754712578 [ https://nmbsignatures.s3.us-east-2.amazonaws.com/banner.jpg ] *This e-mail and any attachments are proprietary to NMB PLC. It is only addressed to the addressee and NMB PLC shall not be responsible for any further publication of the contents of this e-mail.*

Hi, Great that you want to subscribe to Kafka's users mailing list! Subscription does not work by sending an e-mail to the list. Hence, you are not subscribed to the list and will not get any e-mails sent to the list. To subscribe to the mailing list please follow the instruction at https://kafka.apache.org/contact The website under that link states that you need to send an e-mail to users-subscribe@kafka.apache.org to subscribe to the users mailing list. Best, Bruno On 21.09.25 06:08, 彩彩 wrote: > Subject: Subscription request to Kafka users mailing list > > > 彩彩 > 3360776475@qq.com

Subject: Subscription request to Kafka users mailing list 彩彩 3360776475@qq.com

Hi team, What's the current status of the Java 25 support for the project? Is it safe to deploy Kafka and Connect using a Java 25 runtime? Thank you

Hi Team We have the requirement to consume the messages from the Kafka Topics in WebSphere. To implement this requirement I need Apache Kafka JCA resource adapter which I can import to WebSphere and create the required configuration. Please help me with RAR file.

Hi Team We have the requirement to consume the messages from the Kafka Topics in WebSphere. To implement this requirement I need Apache Kafka JCA resource adapter which I can import to WebSphere and create the required configuration. Please help me with RAR file. Sent from my iPhone

Hi Kota, Thanks for your questions! > - Are there any circumstances where the target (MSK) offsets could be > overwritten by smaller source offsets while consumers are already running > on MSK? > - I assume that this kind of behavior is not intended by design. No, as soon as you start the first consumer on the target cluster, MM2 offset syncing will stop for that group and offsets can only be changed by the target consumers. This is a property enforced by the consumer group coordinator, not a MM2 feature specifically. However, it is extremely likely that the target consumer will get some messages that already had offsets committed by the source consumers, even if you stop the source consumers first. Your consumer application should be able to tolerate this re-delivery if you want to do a live/staged migration (consumers before producers) like you described. > - Will new messages written to onpre-kafka continue to replicate to MSK > ...

Summary I'm migrating from an on-prem Apache Kafka cluster ("onpre-kafka") to an AWS MSK cluster using MirrorMaker 2 (MM2). After enabling one-way replication (onpre-kafka → MSK), I plan to cut consumers over to MSK while producers still write to onpre-kafka. I'd like to confirm the exact offset semantics. Topology - Source: onpre-kafka - Target: MSK - Replication: MM2 (MirrorSourceConnector + MirrorHeartbeatConnector + MirrorCheckpointConnector) - Direction: source → target (one-way) - Producers: continue writing to onpre-kafka - Consumers: switch from onpre-kafka to MSK after offset translation Scenario 1. MM2 replicates topics from onpre-kafka to MSK. 2. I start consuming from MSK (consumers move), while producers still write to onpre-kafka. 3. MSK consumer offsets advance (e.g., 100 → 102). onpre-kafka consumer group offsets remain at 100. Questions - Are there any circumstances where the ta...

Hello Kafka users, developers and client-developers, This is the third candidate for release of Apache Kafka 4.0.1. Since this is a patch version it contains the bug fixes outlined in the release notes below! Release notes for the 4.0.1 release: https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc2/RELEASE_NOTES.html *** Please download, test and vote by Wednesday September 17. Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc2/ * Docker release artifacts to be voted upon: apache/kafka:4.0.1-rc2 apache/kafka-native:4.0.1-rc2 * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc2/javadoc/ * Tag to be voted upon (off 4.0 branch) is the 4.0.1 tag: https://github.com/apa...

Hi Apache Kafka Team, This is to request you to kindly answer my small query. Is it possible to configure Controller of a Kafka cluster running in Kraft mode with SASL_SSL and Oauthbearer mechanism? Scenario: I am using Apache Kafka version 2.13_3.8.1. While trying to configure a controller with SASL_SSL with Oauthbearer mechanism in Kafka running in kraft mode, I am getting error "An internal error occurred while retrieving token from callback handler." While the same credentials and configuration works for Broker in SASL_SSL with Oauthbearer mechanism (controller in Plaintext for this). If this is allowed, then kindly can you share the configuration for the controller? Looking forward to hearing from you. Regards, Aishwarya Saha

Hi the doc here https://kafka.apache.org/39/documentation.html#java didn't mention jdk21 but 3.9.1 does support jdk21. Also, the official docker image here uses jdk21 for kafka 3.9.1 https://hub.docker.com/layers/apache/kafka/3.9.1/images/sha256-5862db4a63a6dd7d46fd14771b10a1b39e069c2c47f17d8e4640f960720a0ead so can you please verify if jdk21 is supported on kafka 3.9.1 or not? If yes is it possible to update the doc mentioning jdk21? Thanks Regards, Nirjas

Hi, When using the Kafka 4.0.0 client, we could enable shared consumers by providing the following properties in an embedded Kafka environment. For example, "unstable.api.versions.enable=true", "group.coordinator.rebalance.protocols=classic,share", "share.coordinator.state.topic.replication.factor=1", "share.coordinator.state.topic.min.isr=1" When upgrading to Kafka 4.1.0, the same tests work with just the last two properties. Is that expected? (Basically, I don't need to set unstable.api.versions.enable or group.coordinator.rebalance.protocols in 4.1.0). The documentation shows something like setting share.version to 1 using the kafka-features.sh script? I am curious why the embedded Kafka cluster worked without going through this procedure. I really appreciate any help you can provide. Soby Chacko

The Apache Kafka community is pleased to announce the release for Apache Kafka 4.1.0 The release brings many new features and improvements: - Mechanism for plugin to register metrics - Allow running multiple versions of connector plugins - Queue are now in preview access - New Streams rebalance protocol in early access - Support for OAuth jwt-bearer grant type - Improved metadata replication All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kafka/4.1.0/RELEASE_NOTES.html An overview of the release can be found in our announcement blog post: https://kafka.apache.org/blog#apache_kafka_410_release_announcement You can download the source and binary release from: https://kafka.apache.org/downloads#4.1.0 --------------------------------------------------------------------------------------------------- Apache Kafka is a distributed streaming platform with four core APIs: ** The Producer API allows an applicati...

Hi, +1 (non-binding). I have tried our Strimzi test container on AArch64. Tests passed without any problem. Thanks. Cheers, Maros.

+1 (non-binding). I did some smoke testing with the staged binaries, and I also ran the Strimzi test suite against the 4.0.1 release. All seems to work fine. Thanks for running the release. Jakub On Tue, Aug 19, 2025 at 5:43 PM Christo Lolov < christololov@gmail.com > wrote: > Hello Kafka users, developers and client-developers, > > This is the second candidate for release of Apache Kafka 4.0.1. > > Since this is a patch version it contains the bug fixes outlined in the > release notes below! > > Release notes for the 4.0.1 release: > https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc1/RELEASE_NOTES.html > > *** Please download, test and vote by Tuesday August 26. > > Kafka's KEYS file containing PGP keys we use to sign the release: > https://kafka.apache.org/KEYS > > * Release artifacts to be voted upon (source and binary): > https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc1/ > ...

This vote passes with 8 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Bill * Justine * Luke Committers: * Andrew Community: * Paolo * Kuan-Po * Federico * Jiunn-Yang 0 votes * No votes -1 votes * No votes Vote thread: https://lists.apache.org/thread/f35x3fr77l4kfr4hdf8r8kvbkp7j4thd I'll continue with the release process and the release announcement will follow in the next few days. Mickael