Kafka

Hi All, There are 2 types of arrays specified in the Kafka protocol documentation : ARRAY and COMPACT_ARRAY. But in the protocol details for the different messages, it does not explicitly specify if the array type is which one of the above. For example, the BNF grammar for the section for MetadataRequest API is as below : Metadata Request (Version: 0) => [topics] topics => name name => STRING What is the type of [topics] ? Is it ARRAY or COMPACT_ARRAY ? After playing around with the protocol using some tests, I think for Version:0 of this API request, the broker expects this to be of type ARRAY. But for higher versions, say v9, COMPACT_ARRAY is expected. I think the protocol really needs to be explicit and is lacking in this respect. Regards, Neeraj

Hello, I am using kafka server 3.4.0 along with flink. Kafka server and Flink are installed on a 48 core , 252GB box. My use case is as follows - 8 Kafka producers writing events at 200K per second to kafka topic "Event" with 20 partitions, source for flink --> Flink processing rules that read from Event topic and write to Alert topic --> kafka topic "Alert" with 20 partitions, sink for flink. It was all good until we started seeing that flink kafka consumer for Event topic getting timed out frequently as kafka responds quite late to the fetch requests. I am not able to figure out the reason why kafka takes a lot of time to process this FETCH request randomly. Is there a configuration that I must look at or nay other log that I must check to figure out whats going on? When everything is fine, kakfa takes only a few milliseconds to process the fetch requests. Timeout of kafka consumer at the flink side is 30 secs and the consumer thread blocks until ...

Hello Kafka users, developers and client-developers, This is the third candidate for the release of Apache Kafka 3.6.0. Some of the major features include: * KIP-405 : Kafka Tiered Storage * KIP-868 : KRaft Metadata Transactions * KIP-875: First-class offsets support in Kafka Connect * KIP-898: Modernize Connect plugin discovery * KIP-938: Add more metrics for measuring KRaft performance * KIP-902: Upgrade Zookeeper to 3.8.1 * KIP-917: Additional custom metadata for remote log segment Release notes for the 3.6.0 release: https://home.apache.org/~satishd/kafka-3.6.0-rc2/RELEASE_NOTES.html *** Please download, test and vote by Tuesday, October 3, 12pm PT Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://home.apache.org/~satishd/kafka-3.6.0-rc2/ * Maven artifacts to be voted upon: https://repository.apache.org/content/groups/staging/org/apa...

In general, Kafka Streams should keep running. Can you inspect the logs to figure out why it's going into ERROR state to begin with? Maybe you need to increase/change some timeouts/retries configs. The stack trace you shared, is a symptom, but not the root cause. -Matthias On 9/21/23 12:56 AM, Debraj Manna wrote: > I am using Kafka broker 2.8.1 (from AWS MSK) with Kafka clients and Kafka > stream 3.5.1. > > I am observing that whenever some rolling upgrade is done on AWS MSK our > stream application reaches an error state. I get the below exception on > trying to query the state store > > caused by: java.lang.IllegalStateException: KafkaStreams is not running. > State is ERROR. > at > org.apache.kafka.streams.KafkaStreams.validateIsRunningOrRebalancing(KafkaStreams.java:381) > at > org.apache.kafka.streams.KafkaStreams.queryMetadataForKey(KafkaStreams.java:1663) > at > org.springframework.c...

For the config you provide, data loss should not happen (as long as you don't allow for unclean leader election, which is disabled by default). But you might be subject to unavailability for some partitions if a broker fails. -Matthias On 9/17/23 7:49 AM, 陈近南 wrote: > Hello, > Can a message avoid loss occur in Kafka. For example, my config is: > > > Producer > retries = Integer.MAX_VALUE > request.required.acks=-1 > > > Broker > replication.factor >= 2 > min.insync.replicas > 1 > log.flush.interval.messages=1 > > > Consumer > enable.auto.commit = false > > Can it avoid loss message occur in Kafka, if can not, why? and does exist other MQ can do avoid? > > > Best regards, > Chen >

Hi Venkat, are you planning to use Open source Apache Kafka Or Confluent? what is your use case apart from streaming? regards, Sunil. On Fri, 29 Sep 2023 at 12:27 PM, ANANTHA VENKATA GANJAM <g.vijaya@tcs.com.invalid> wrote: > TCS Confidential > > Hi Team, > > We are planning to set up Lab environment for Kafka in TCS. Please guide > us on the next steps. > > > Thanks & Regards, > Vijaya Sri > > > > TCS Confidential > =====-----=====-----===== > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply ...

TCS Confidential Hi Team, We are planning to set up Lab environment for Kafka in TCS. Please guide us on the next steps. Thanks & Regards, Vijaya Sri TCS Confidential =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

We have an existing cluster that is CentOS 7 and we need to migrate to Ubuntu in the next 8 months. We are unable to build a new cluster and deploy because of the complexities in clients (producers and consumers). We would like to either. 1. Replace individual CentOS nodes one at a time (both broker and ZK) with Ubuntu nodes. Has anyone run mixed OS nodes in a cluster? This would only be for a short period of time. 2. Just add Ubuntu nodes and migration partitions. Same issue...we would be running mixed OS nodes at the same time. The bottom line is can I run (for a short time) a mix of CentOS7 and Ubuntu nodes in the same cluster (ZK as well) -- Gene Robichaux Senior Director, Platform Engineering Match/Affinity

Thank you, Yash. That additional documentation helps to further my understanding. In case it helps in any way, I am currently setting the rest.advertised.host.name , and listener properties to a private IP address that is resolvable within each data center. However, each data center can only communicate with each other using a load balancer. Is there any configuration I can set to help with this setup? For example, if the worker sends the request to the load balancer of the data center where the leader resides, I believe that it would work network-wise Thank you again for taking the time to help. ---- On Tue, 26 Sep 2023 07:44:06 -0400 Yash Mayya < yash.mayya@gmail.com > wrote --- Hi Yeikel, > To clarify, who initiates the step that assigns a > connector to a specific worker? If this process > is controlled by the leader, wouldn't it result in a > failure to assign tasks to workers with whom it > cannot communicate? ...

Hi Jose, Sounds good to me. Let's have further discussion in JIRA/PR, and target to v3.6.1/v3.7.0. Thanks. Luke On Tue, Sep 26, 2023 at 1:35 AM José Armando García Sancio <jsancio@confluent.io.invalid> wrote: > On Sat, Sep 23, 2023 at 3:08 AM Luke Chen < showuon@gmail.com > wrote: > > > > Hi Satish, > > > > I found the current KRaft implementation will have "split brain" issue > when > > network partition happens, which will cause inconsistent metadata > returned > > from the controller. > > Filed KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 > > for > > this issue, and PR < https://github.com/apache/kafka/pull/14428 > is ready > > for review. > > > > Even though this is not a regression issue (this has already existed > since > > the 1st release of KRaft feature), I think this is an important issue > ...

On Sat, Sep 23, 2023 at 3:08 AM Luke Chen < showuon@gmail.com > wrote: > > Hi Satish, > > I found the current KRaft implementation will have "split brain" issue when > network partition happens, which will cause inconsistent metadata returned > from the controller. > Filed KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 > for > this issue, and PR < https://github.com/apache/kafka/pull/14428 > is ready > for review. > > Even though this is not a regression issue (this has already existed since > the 1st release of KRaft feature), I think this is an important issue since > KRaft is announced production ready. > Not sure what other people's thoughts are. Thanks for the report and PR Luke. This looks related to this issue: https://issues.apache.org/jira/browse/KAFKA-13621 Do you agree? We can move our conversation to those issues but I also agree that I don't think this issu...

Kudos to Nikhil. your explanation adds to my knowledge. 🙏 On Mon, 25 Sep 2023 at 7:16 PM, Nikhil Srivastava < nikhilsrivastava4442@gmail.com > wrote: > Hi Yeikel, > > Sharing my two cents. Would let others chime in to add to this. > > Based on my understanding, if connect workers (which are all part of the > same cluster) can communicate with the kafka brokers (which happens to be > the Group Coordinator and facilitates Connect Leader Election via Group > Membership Protocol), then only 1 connect worker will be elected as leader > amongst all others in the cluster. Outside of that, I believe a bunch of > REST calls to connect workers are forwarded to the connect leader (if the > REST request lands on a connect worker which isn't a leader). In case of a > non-retriable network partition between the non-leader worker and leader > worker, those REST requests will fail. I'm referring to REST requests like > CR...

Hi Yeikel, Sharing my two cents. Would let others chime in to add to this. Based on my understanding, if connect workers (which are all part of the same cluster) can communicate with the kafka brokers (which happens to be the Group Coordinator and facilitates Connect Leader Election via Group Membership Protocol), then only 1 connect worker will be elected as leader amongst all others in the cluster. Outside of that, I believe a bunch of REST calls to connect workers are forwarded to the connect leader (if the REST request lands on a connect worker which isn't a leader). In case of a non-retriable network partition between the non-leader worker and leader worker, those REST requests will fail. I'm referring to REST requests like CREATE / UPDATE / DELETE. Hope this helps a little. Thanks, -Nikhil On Sun, 24 Sept 2023 at 06:36, Yeikel Santana < email@yeikel.com > wrote: > Hello everyone,I'm currently designing a new Kafka Connect cluster, ...

Hi Alex We currently do not have plans but we can definitely use a helping hand with our website content ( e.g. our blogs are currently written in html which is quite painful to manage :) ). An RSS friendly blog would be a great addition. It would be great if the community could help us in adding this feature. -- Divij Vaidya On Fri, Sep 22, 2023 at 8:13 PM Alexander Kilian < alexander.kilian@vermiculus.se > wrote: > > Hi all, > > The old Kafka blog on https://blogs.apache.org/kafka was RSS friendly and could be subscribed to in e.g. Feedly. > > The new Kafka blog on https://kafka.apache.org/blog does not support that anymore. Any plans of adding RSS functionality again? > > Thanks > > Alex

Hi all, I have noticed that the latest update on KIP-833 < https://cwiki.apache.org/confluence/display/KAFKA/KIP-833%3A+Mark+KRaft+as+Production+Ready > (marking KRaft as production ready) was on April 2023. Because I am interested in the plan related to the listed Kafka versions for deprecating ZooKeeper and having only KRaft support, I was wondering if after 5 months (since the last update) the described plan is still valid or it needs an update. Thanks, Paolo. -- Paolo Patierno *Senior Principal Software Engineer @ Red Hat**Microsoft MVP on **Azure* Twitter : @ppatierno < http://twitter.com/ppatierno > Linkedin : paolopatierno < http://it.linkedin.com/in/paolopatierno > GitHub : ppatierno < https://github.com/ppatierno >

Hi Satish, I did the following to verify the release: - Verified signature and checksum - Built from source with Java 17 and Scala 2.13 - Ran all unit and integration tests - Spot checked release notes and documentation - Ran a custom client using staging artifacts on a 3-nodes cluster - Tested tiered storage with one of the available RSM implementations +1 (non binding) Thanks Fede On Sun, Sep 24, 2023 at 8:49 AM Luke Chen < showuon@gmail.com > wrote: > > Hi Satish, > > I verified with: > 1. Ran quick start in KRaft for scala 2.12 artifact > 2. Making sure the checksum are correct > 3. Browsing release notes, documents, javadocs, protocols. > > I filed KAFKA-15491 < https://issues.apache.org/jira/browse/KAFKA-15491 >for > log output improvement while testing stream application. > It won't be blocker in v3.6.0. > > For KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 >, I'm ...

Hi Satish, I verified with: 1. Ran quick start in KRaft for scala 2.12 artifact 2. Making sure the checksum are correct 3. Browsing release notes, documents, javadocs, protocols. I filed KAFKA-15491 < https://issues.apache.org/jira/browse/KAFKA-15491 >for log output improvement while testing stream application. It won't be blocker in v3.6.0. For KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 >, I'm fine if we decide to fix it in v3.6.1/v3.7.0. +1 (binding) from me. Thank you. Luke On Sun, Sep 24, 2023 at 3:38 AM Ismael Juma < me@ismaeljuma.com > wrote: > Given that this is not a regression and there have been no reports for over > a year, I think it's ok for this to land in 3.6.1. > > Ismael > > On Sat, Sep 23, 2023 at 9:32 AM Satish Duggana < satish.duggana@gmail.com > > wrote: > > > Thanks Luke for reporting KRaft issue[1]. > > > > I am not sure whether...

Given that this is not a regression and there have been no reports for over a year, I think it's ok for this to land in 3.6.1. Ismael On Sat, Sep 23, 2023 at 9:32 AM Satish Duggana < satish.duggana@gmail.com > wrote: > Thanks Luke for reporting KRaft issue[1]. > > I am not sure whether it is a release blocker for 3.6.0. Need input > from other KRaft experts also to finalize the decision. Even if we > adopt a fix, do not we need to bake it for some time before it is > pushed to production to avoid any regressions as this change is in the > critical paths? > > 1. https://issues.apache.org/jira/browse/KAFKA-15489 > > Thanks, > Satish. > > On Sat, 23 Sept 2023 at 03:08, Luke Chen < showuon@gmail.com > wrote: > > > > Hi Satish, > > > > I found the current KRaft implementation will have "split brain" issue > when > > network partition happens, which will cause i...

Thanks Luke for reporting KRaft issue[1]. I am not sure whether it is a release blocker for 3.6.0. Need input from other KRaft experts also to finalize the decision. Even if we adopt a fix, do not we need to bake it for some time before it is pushed to production to avoid any regressions as this change is in the critical paths? 1. https://issues.apache.org/jira/browse/KAFKA-15489 Thanks, Satish. On Sat, 23 Sept 2023 at 03:08, Luke Chen < showuon@gmail.com > wrote: > > Hi Satish, > > I found the current KRaft implementation will have "split brain" issue when > network partition happens, which will cause inconsistent metadata returned > from the controller. > Filed KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 > for > this issue, and PR < https://github.com/apache/kafka/pull/14428 > is ready > for review. > > Even though this is not a regression issue (this has already existed sinc...

Hi Satish, I found the current KRaft implementation will have "split brain" issue when network partition happens, which will cause inconsistent metadata returned from the controller. Filed KAFKA-15489 < https://issues.apache.org/jira/browse/KAFKA-15489 > for this issue, and PR < https://github.com/apache/kafka/pull/14428 > is ready for review. Even though this is not a regression issue (this has already existed since the 1st release of KRaft feature), I think this is an important issue since KRaft is announced production ready. Not sure what other people's thoughts are. Thank you. Luke On Thu, Sep 21, 2023 at 6:33 PM Josep Prat <josep.prat@aiven.io.invalid> wrote: > Hi Satish, > > I ran the following validation steps: > - Built from source with Java 11 and Scala 2.13 > - Verified Signatures and hashes of the artifacts generated > - Navigated through Javadoc including links to JDK classes > - Run the uni...