Kafka

hi Luke Thanks for raising this discussion > Luke Chen < showuon@gmail.com > 於 2025年8月1日 上午9:42 寫道： > > 1. Notify users using kafka v4.0.0 about this typo bug, in case you are > trying to grep state logs using regex or something. IIRC, we had a similar discussion about log prefix. "Should we keep the log output format, such as the prefix, for users who might grep logs using regex" That is a gray area, but I agree that is not a part of public APIs. Best, Chia-Ping

Hi all, Since Kafka 4.0.0, there is a typo in the state-change.log file name after the log file rolled out. The file name will be named with "sta*g*e-change.xxx", not "sta*t*e-change.xxx". Ex: "stage-change.log.2025-07-01-20" So, in the logs folder, you should see files mixed with "stage-change.log.xxx" and "state-change.log", like this: ``` > ls logs/sta* logs/stage-change.log.2025-07-01-19 logs/stage-change.log.2025-07-01-20 logs/state-change.log ``` In this mail, I'd like to: 1. Notify users using kafka v4.0.0 about this typo bug, in case you are trying to grep state logs using regex or something. 2. Could we merge this PR < https://github.com/apache/kafka/pull/20269 > even though this will change the user interface? I think this is definitely a bug fix, not a public API change. So no KIP needed. Any objections? 3. We should include this patch in kafka v4.0.1. cc @Christo Lolov < christololo...

Hey everyone, I am having some issues I am very confused about, and I hope someone can help me understand what I am doing wrong. I use consecutive numbers as values for some messages. Those messages are then sent to Kafka via a Produce Request. The target topic has three partitions, and the messages are spread among them. When I issue a Fetch Request, there is always only one batch with one message returned. I also checked the latest offsets beforehand, and they show that there is more than one message inside the given partition. Reading into the various settings, I also ensured that I have the following set in the Fetch request * fetch.min.bytes = 100 * 1024, * fetch.max.bytes = 1024 ** 2, * max.partition.fetch.bytes = 1024 ** 2, * fetch.max.wait.ms = 2000 Additionally, I have set * message.max.bytes = 1000012 As the messages are clearly much smaller than fetch.min.bytes, and they are already stored in the logs, I would expect to receive more th...

Good day. I have configured Kafka to run on docker swarm, when I try to use Python to send messages, I am being presented with the error messages such as the ones below in rapid succession, non-stop. What changes should I make so that I am able to send messages to the broker? ERROR:kafka.conn:Connect attempt to <BrokerConnection node_id=1 host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]> returned error 111. Disconnecting. INFO:kafka.conn:<BrokerConnection node_id=1 host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]>: Closing connection. KafkaConnectionError: 111 ECONNREFUSED WARNING:kafka.client:Node 1 connection failed -- refreshing metadata INFO:kafka.conn:<BrokerConnection node_id=1 host=localhost:9092 <connecting> [IPv4 ('127.0.0.1', 9092)]>: connecting to localhost:9092 [('127.0.0.1', 9092) IPv4] ERROR:kafka.conn:Connect attempt to <BrokerConnection node_id=1 host=loc...

Sorry for the broken link, retrying-  https://issues.apache.org/jira/browse/KAFKA-19552   On Saturday, July 26, 2025, 11:02 PM, Mcs Vemuri < mcsvemuri@yahoo.com > wrote: Hello, Can anyone please check this bug - ASF JIRA  Unclean leader election failing to happen- while being desired-  leaves cluster broken when brokers leave suddenly and indefinitely   | | | ASF JIRA | | | Sent from Yahoo Mail for iPhone

Hello, Can anyone please check this bug - ASF JIRA  Unclean leader election failing to happen- while being desired-  leaves cluster broken when brokers leave suddenly and indefinitely   | | | ASF JIRA | | | Sent from Yahoo Mail for iPhone

Hello Kafka users, developers and client-developers, This is the first candidate for release of Apache Kafka 4.0.1. Since this is a patch version it contains the bug fixes outlined in the release notes below! Release notes for the 4.0.1 release: https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc0/RELEASE_NOTES.html *** Please download, test and vote by Wednesday July 30. Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc0/ * Docker release artifacts to be voted upon: apache/kafka:4.0.1-rc0 apache/kafka-native:4.0.1-rc0 * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apache/kafka/ * Javadoc: https://dist.apache.org/repos/dist/dev/kafka/4.0.1-rc0/javadoc/ * Tag to be voted upon (off 4.0 branch) is the 4.0.1 tag: https://github.com/apache/k...

Re, Thank you very much for your help! I wandered around for several days. It worked by adding to a clickhouse connector creation http request: * "consumer.override.ssl.truststore.location": "/opt/kafka/kafka.ca.pem", "consumer.override.ssl.truststore.type": "PEM", "consumer.override.ssl.keystore.type": "PEM", "consumer.override.ssl.keystore.location": "/opt/kafka/kafka.client.p8.pem", "consumer.override.ssl.client.auth": "required" * /Domantas On Wed, Jul 23, 2025 at 4:59 PM Chris Egerton < fearthecellos@gmail.com > wrote: > Hi Domantas, > > For sink connectors, you'll need to add all SSL-related properties either > to your Connect worker file prefixed with "consumer.", or to your > individual connector files prefixed with "consumer.override.". > > If you're using the DLQ fe...

Hello Sébastien, thanks for reaching out. The fix was not back ported to 3.x branches, because it is very unlikely that there will be another 3.x bug fix release. Kafka community usually only does bug-fix releases for the last minor release, and there was already 3.9.1. With 4.1.0 being in the process or getting release, we would only do another 3.9.2 bug-fix release if there is some highly critical issue, and user cannot upgrade to a newer 4.x version. A 3.8.2 or older version bug-fix release is even more unlikely. Thus, my recommendation would be, to wait for 4.0.1 ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.0.1 ) or 4.1.0 ( https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+4.1.0 ) and upgrade your KS app to either one version to get the fix. -Matthias On 7/21/25 10:22 AM, Sébastien wrote: > Hi everyone, > > I'm upgrading my old Kafka Stream application currently in 2.6. to 3.8.1 >...

Hi Domantas, For sink connectors, you'll need to add all SSL-related properties either to your Connect worker file prefixed with "consumer.", or to your individual connector files prefixed with "consumer.override.". If you're using the DLQ feature, you'll also need to do the same but with "admin." (worker file) or "admin.override." (connector files). And, for source connectors, do the same but with "producer." (worker file) / "producer.override." (connector files). Cheers, Chris On Wed, Jul 23, 2025, 08:31 Domantas Spečiūnas <domantas.speciunas@aardvark.technology.invalid> wrote: > HI, > > I have issue connecting to to kafka with SSL, tried a lot of options and > stuck, maybe someone can suggest what is wrong here, on clickhouse side > everything is fine: > > [root@server.tld01 config]# cat connect-distributed.properties | grep -v > ^# | grep -v '^...

HI, I have issue connecting to to kafka with SSL, tried a lot of options and stuck, maybe someone can suggest what is wrong here, on clickhouse side everything is fine: [root@server.tld01 config]# cat connect-distributed.properties | grep -v ^# | grep -v '^$' group.id =connect-cluster key.converter=org.apache.kafka.connect.json.JsonConverter value.converter=org.apache.kafka.connect.json.JsonConverter key.converter.schemas.enable=true value.converter.schemas.enable=true offset.storage.topic=connect-offsets offset.storage.replication.factor=1 config.storage.topic=connect-configs config.storage.replication.factor=1 status.storage.topic=connect-status status.storage.replication.factor=1 offset.flush.interval.ms =10000 listeners= HTTP://127.0.0.1:8083 plugin.path=/opt/kafka/connectors ssl.keystore.location=/opt/kafka.client.p8.pem ssl.keystore.type=PEM bootstrap.servers=server.tld01:9091 security.protocol=SSL ssl.truststore.type=PEM ssl.truststore.loca...

Hi, Curious if anyone has thought about using Kafka Queues (KIP-932) with Kafka Connect in the future? I.e. autoscaling connect tasks? There could be some Connect use cases where order isn't as important as keeping up with load spikes, maybe. Regards, Paul Brebner NetApp Instaclustr Technology Evangelist Read my blogs at https://www.instaclustr.com/paul-brebner/ Follow me on https://www.linkedin.com/in/paul-brebner-0a547b4/

hi If anyone wants to test 4.1.0-RC1 without the backport fix, please simply configure the jdk_version to use another image. For example: jdk_version="openjdk:17-bullseye" /bin/bash tests/docker/run_tests.sh Best, Chia-Ping Josep Prat <josep.prat@aiven.io.invalid> 於 2025年7月22日 週二 下午10:56寫道： > Hi Mickael, > I don't think we need a new RC just because of these files. Technically > speaking these files are part of the source tarball (which we release), but > as they are test files, I wouldn't consider them as a blocker that would > grant a new RC. > > Best, > > On Tue, Jul 22, 2025 at 4:51 PM Mickael Maison < mickael.maison@gmail.com > > wrote: > > > Hi, > > > > Quick update regarding the system tests. My environment is back but > > now we're hitting https://issues.apache.org/jira/browse/KAFKA-19501 . > > > > Debian buster which is used in the system tests ...

Hi Mickael, I don't think we need a new RC just because of these files. Technically speaking these files are part of the source tarball (which we release), but as they are test files, I wouldn't consider them as a blocker that would grant a new RC. Best, On Tue, Jul 22, 2025 at 4:51 PM Mickael Maison < mickael.maison@gmail.com > wrote: > Hi, > > Quick update regarding the system tests. My environment is back but > now we're hitting https://issues.apache.org/jira/browse/KAFKA-19501 . > > Debian buster which is used in the system tests is out of support and > it's apt repository has now been archived. It worked till last week > but now it's gone so we get an error when building the Docker image. > We had only updated trunk to use bullseye. I've just backported the > fix to 4.1: > https://github.com/apache/kafka/commit/d5a3acda897890379edb8819f15b144fcea564b0 > > The fix is only in tests/docker...

Hi all,  I'm currently running a Kafka Connect cluster within a Kubernetes environment (OpenShift). Since I couldn't find much documentation on how to integrate, I integrated using the OpenTelemetry Java Agent [1], which seems to work. That said, I still lack visibility into many Kafka-specific metrics, and I'm not sure if this is the best approach in general.  If anyone has experience with this setup or can recommend a better way to monitor Kafka Connect in Kubernetes using OpenTelemetry, I'd really appreciate your suggestions. [1]  https://opentelemetry.io/docs/zero-code/java/agent/  

Hi, Quick update regarding the system tests. My environment is back but now we're hitting https://issues.apache.org/jira/browse/KAFKA-19501 . Debian buster which is used in the system tests is out of support and it's apt repository has now been archived. It worked till last week but now it's gone so we get an error when building the Docker image. We had only updated trunk to use bullseye. I've just backported the fix to 4.1: https://github.com/apache/kafka/commit/d5a3acda897890379edb8819f15b144fcea564b0 The fix is only in tests/docker/Dockerfile and tests/docker/ducker-ak. My understanding is that these files are not part of any of the release artifacts. So I'm considering keeping the RC1 artifacts and running the system tests with the fix. Let me know if my understand is incorrect, or if you'd prefer building a new RC regardless. Obviously if we find issues with RC1, we'll do another RC anyway. Thanks, Mickael On Mon, Jul ...

Hi everyone, I'm upgrading my old Kafka Stream application currently in 2.6. to 3.8.1 version. But I'm facing a bug where some KStream-GlobalKTable left join doesn't work anymore and instead Drop records, and that's a regression from my 2.6.1 version. What I have is a Kstream With K1,V1 Avro records (Key and value) and I want to leftJoin a GlobalKTable that has K2,V2 Avro records. K2 can be extracted from V1 record (a classic Foreign Key) What I am doing is that I use a KeyValueMapper to map V1 to K2 If I can't build K2 because necessary values in V1 are null, since I can't build K2 (Avro Object) with null values I return null from the mapper and it just do a Left Outer Join and record is not dropped (actual behavior in 2.6). But code introduced in KIP-962 change that, and new behavior in 3.7+ versions is to drop records if mappedKey is null I'm sure that is an unwanted regression since the KIP-692 was supposed to relax non-null req...

Hi everyone, I'm upgrading my old Kafka Stream application currently in 2.6. to 3.8.1 version. But I'm facing a bug where some KStream-GlobalKTable left join doesn't work anymore and instead Drop records, and that's a regression from my 2.6.1 version. What I have is a Kstream With K1,V1 Avro records (Key and value) and I want to leftJoin a GlobalKTable that has K2,V2 Avro records. K2 can be extracted from V1 record (a classic Foreign Key) What I am doing is that I use a KeyValueMapper to map V1 to K2 If I can't build K2 because necessary values in V1 are null, since I can't build K2 (Avro Object) with null values I return null from the mapper and it just do a Left Outer Join and record is not dropped (actual behavior in 2.6). But code introduced in KIP-962 change that, and new behavior in 3.7+ versions is to drop records if mappedKey is null I'm sure that is an unwanted regression since the KIP-692 was supposed to relax non-null req...

Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 4.1.0. The release brings many new features and improvements: - Mechanism for plugin to register metrics - Allow running multiple versions of connector plugins - Queue are now in preview access - New Streams rebalance protocol in early access - Support for OAuth jwt-bearer grant type - Improved metadata replication Release notes for the 4.1.0 release: https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc1/RELEASE_NOTES.html *** Please download, test and vote by Friday July 25. Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://dist.apache.org/repos/dist/dev/kafka/4.1.0-rc1/ * Docker release artifacts to be voted upon: apache/kafka:4.1.0-rc1 apache/kafka-native:4.1.0-rc1 * Maven artifacts to be voted upon: https://repository.ap...

Hello, We are experiencing stability issues in our Kafka architecture during chunked file transfers, especially during load spikes or after broker restarts. Here are the details: ------------------------------ 📦 *Architecture*: - KafkaJS v2.2.4 used as both *producer and consumer*. - Kafka cluster with *3 brokers*, connected to a *shared NAS*. - Files are *split into 500 KB chunks* and sent to a Kafka topic. - A dedicated consumer reassembles files on the NAS. - Each file is assigned to the *topic/broker of the first chunk* to preserve order. ------------------------------ ❌ *Observed issues*: 1. *Request Produce(version: 7) timed out error during load spikes*: - For about *15 minutes*, KafkaJS producers fail with the error: *Request Produce(key: 0, version: 7) timed out* - This generally occurs *during traffic spikes*, typically when *multiple files are being ...

Congratulations Lucas! On Mon, Jul 14, 2025 at 6:02 PM Matthias J. Sax < mjsax@apache.org > wrote: > Dear all, > > The Project Management Committee (PMC) for Apache Kafka is pleased to > announce Lucas Brutschy as a new PMC member. > > Lucas has been a Kafka committer since September 2023. He has remained > very active in the community since then, and it's my pleasure to > announce that he has accepted our invitation to become a member of the > Kafka PMC. > > Congratulation Lucas! > > > Matthias, on behalf of the Apache Kafka PMC >

Dear all, The Project Management Committee (PMC) for Apache Kafka is pleased to announce Lucas Brutschy as a new PMC member. Lucas has been a Kafka committer since September 2023. He has remained very active in the community since then, and it's my pleasure to announce that he has accepted our invitation to become a member of the Kafka PMC. Congratulation Lucas! Matthias, on behalf of the Apache Kafka PMC

cc users ---------- Forwarded message --------- From: Federico Valeri < fedevaleri@gmail.com > Date: Mon, Jul 14, 2025 at 2:09 PM Subject: MX4j support To: < dev@kafka.apache.org > Hello devs, Kafka has support for MX4j, that can be used to expose JMX beans over HTTP. https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/KafkaRaftServer.scala#L91 This feature is not documented and disabled by default. To enable it, you have to pass -Dkafka_mx4jenable=true and add the required JARs to the classpath. Activity on MX4J project has ceased and last release was 12/10/2006. We have an 8 years old post on Kafka users ML asking if there is any user, but no one replied. https://en.wikipedia.org/wiki/MX4J https://users.kafka.apache.narkive.com/4BMpuEMX/is-anyone-using-mx4j-loader My question is: should we deprecate and remove? I guess that would require a KIP. I have some time to work on this if there is agreement. Thanks...

Hi, I think I found something which could be a blocker for the release. I was adding the support for Apache Kafka 4.1.0 to Strimzi and running our tests but some of them are failing with the following error. java.lang.IllegalArgumentException: Version 4.1 is not a valid version. The minimum version is 3.3-IV3 and the maximum version is 4.2-IV1 at org.apache.kafka.server.common.MetadataVersion.lambda$fromVersionString$0(MetadataVersion.java:356) at java.base/java.util.Optional.orElseThrow(Optional.java:403) at org.apache.kafka.server.common.MetadataVersion.fromVersionString(MetadataVersion.java:354) In these tests we use the MetadataVersion class to verify the validity of metadata but it fails when passing something like "4.1" (while they work just fine with all previous releases, i.e. "4.0", "3.9" and so on). The exception, thrown by the fromVersionString method, seems to be related to "4.1" missing as key entr...

Hello Thanks for the response. Your suggestion of setting auto.commit to true and performing commitSync() while using subscribe method did not work. Poll does not return any records. [L4J2-test] 12:21:38 INFO : test.kafka.TestKafkaCluster - Kafka properties: { node.id =1, retry.backoff.max.ms =5000, log.flush.interval.messages=1, reconnect.backoff.max.ms =5000, listeners=CLIENT:// 127.0.0.1:56431 ,INTERNAL:// 127.0.0.1:56432 ,CONTROLLER:// 127.0.0.1:56434 ,EXTERNAL:// 127.0.0.1:56433 , log.flush.interval.ms =1, inter.broker.listener.name =INTERNAL, reconnect.backoff.ms =2000, retry.backoff.ms =2000, bootstrap-server= 127.0.0.1:56431 , retries=10, controller.quorum.voters= 1@127.0.0.1:56434 <mailto: controller.quorum.voters = 1@127.0.0.1:56434 >, zookeeper.connect= 127.0.0.1:56427 , process.roles=broker,controller, controller.listener.names=CONTROLLER, broker.session.timeout.ms =600000, listeners.external.bootstrap.servers=PLAINTEXT:// 127.0.0.1:56433 , connections.max.idle.m...