Kafka

Thanks. I'm aware of ISR but looking for exact logic inside Java client how is this handled so when follower replica, consumer is using, starts lagging / goes out-of-sync. On Mon, Sep 20, 2021 at 2:52 PM Luke Chen < showuon@gmail.com > wrote: > Hi Michał, > I think you want to check the term in Kafka: In Sync Replicas (a.k.a ISR). > You might want to check the "Replication" section in doc here > < https://kafka.apache.org/documentation/#replication >. > > The configuration is this one: * replica.lag.time.max.ms > < http://replica.lag.time.max.ms >* > > https://kafka.apache.org/documentation/#brokerconfigs_replica.lag.time.max.ms > > Hope that helps. > > Thank you. > Luke > > On Mon, Sep 20, 2021 at 8:30 PM Michał Łowicki < mlowicki@gmail.com > wrote: > > > Hey, > > > > When Java client is reading from replica then how is handled the case > when ...

Hi, You could try to use a stream processing library that supports joining multiple streams. Check out https://www.confluent.io/blog/crossing-streams-joins-apache-kafka/ Best, Denis On Mon, Sep 20, 2021 at 6:46 PM Andrew Rakov < rakovandrew812@gmail.com > wrote: > Hello! > > I'm working with Kafka to send messages from one microservice to another. > And I have a problem. It completely stops my work. > > I have a couple of topics with a few partitions. I want to round-robin read > from each topic. I mean to take one message from a topic and go to the > next. But I can't understand how to code this. > > Is it possible at all? Could you advise me on how to do it? > > P.S. In my case consumer reads all messages in a topic and goes to the > next. > > Best regards, > Andrew Rakov > -- Regards, Denis Nutiu

Hi All, I have a kafka cluster running in kubernetes with 3 brokers. I have a topic with 3 partitions and replication factor of 3. Topic is evenly distributed across brokers. I have noticed that there is a huge difference in disk utilization of one partition between replica and leader. Ideally replica and leader partition should have same disk utilization if there is no issue with replication. In my case replication is running fine and under replicated partitions count is 0. what could be the reason of this anomaly ? Thanks, Dhirendra.

Thank you, David, for serving as the release manager for 2.8.1. Great work! And congratulations to the Apache Kafka community! Best regards, Randall On Mon, Sep 20, 2021 at 2:48 AM David Jacot < dajac@apache.org > wrote: > The Apache Kafka community is pleased to announce the release for > Apache Kafka 2.8.1 > > Apache Kafka 2.8.1 is a bugfix release and fixes 49 issues since the 2.8.0 > release. Please see the release notes for more information. > > All of the changes in this release can be found in the release notes: > https://www.apache.org/dist/kafka/2.8.1/RELEASE_NOTES.html > > > You can download the source and binary release (Scala 2.12 and 2.13) from: > https://kafka.apache.org/downloads#2.8.1 > > > --------------------------------------------------------------------------------------------------- > > > Apache Kafka is a distributed streaming platform with four core APIs: > > ...

did you try setting these values in the distributor.properties file? On Mon, Sep 20, 2021 at 12:43 AM Jamie <jamiedd13@aol.co.uk.invalid> wrote: > Hi All, > Has anyone been able to override the producer and consumer values of the > MM2 MirrorSourceConnector as described below? > Many Thanks, > Jamie > > > -----Original Message----- > From: Jamie < jamiedd13@aol.co.uk > > To: users@kafka.apache.org < users@kafka.apache.org > > Sent: Thu, 16 Sep 2021 15:52 > Subject: MM2 - Overriding MirrorSourceConnector Consumer & Producer values > > Hi All, > I've trying to override the properties of the consumer and producer in MM2 > to tune them for high throughput. For example, I'm trying to set the > consumers fetch.min.bytes to 100000. > I'm running MM2 in a dedicated mirror maker cluster ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0#KIP382:MirrorM...

Hello! I'm working with Kafka to send messages from one microservice to another. And I have a problem. It completely stops my work. I have a couple of topics with a few partitions. I want to round-robin read from each topic. I mean to take one message from a topic and go to the next. But I can't understand how to code this. Is it possible at all? Could you advise me on how to do it? P.S. In my case consumer reads all messages in a topic and goes to the next. Best regards, Andrew Rakov

ApacheCon @Home starts tomorrow! Details at https://www.apachecon.com/acah2021/index.html (Note: You're receiving this because you are subscribed to one or more user lists for Apache Software Foundation projects.) We've got three days of great content lined up for you, spanning 14 project communities. And we're very excited about the keynotes, with presentations from David Nalley, Ashley Wolfe, Mark Cox, Alison Parker, and Michael Weinberg. And we'll be hearing from our Platinum sponsors in their keynotes as well! (Schedule is at https://www.apachecon.com/acah2021/tracks/ ) You can still register today, at https://www.apachecon.com/acah2021/register.html We especially want to thank our sponsors, who have made this event possible: Strategic sponsor: Google Platinum sponsors: Huawei, Tencent, Instaclustr, and Apple Gold sponsors: AWS, Aiven, Gradle, Replicated, Red Hat, Baidu, Fiter,...