Kafka

Yes, you can configure replication.factor to match the target cluster. Ryanne On Tue, Jul 12, 2022, 12:44 PM An, Hongguo (CORP) <Hongguo.An@adp.com.invalid> wrote: > Hi: > My source has 5 brokers and all topic has 5 replica, but my target cluster > has only 3 brokers and I can't allocate 2 more, is it possible for MM2 to > change all topic copied to have replica as 3? > > Thanks > Andrew > > > This message and any attachments are intended only for the use of the > addressee and may contain information that is privileged and confidential. > If the reader of the message is not the intended recipient or an authorized > representative of the intended recipient, you are hereby notified that any > dissemination of this communication is strictly prohibited. If you have > received this communication in error, notify the sender immediately by > return email and delete the message and any attachments from your sys...

Hi: My source has 5 brokers and all topic has 5 replica, but my target cluster has only 3 brokers and I can't allocate 2 more, is it possible for MM2 to change all topic copied to have replica as 3? Thanks Andrew This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, notify the sender immediately by return email and delete the message and any attachments from your system.

Hello, Can anyone please help me regarding the below query regarding SSL communication in Kafka: Query: Is there any way to enable the hostname verification for Kafka communication between broker and client without specifying the IP address in SAN? Regards, Deepak From: Deepak Jain Sent: 08 July 2022 01:23 To: Luke Chen < showuon@gmail.com > Cc: users@kafka.apache.org Subject: Inquiry about using SSL encryption and SASL authentication for Kafka without specifying IP address in SAN in the CA certificate Hi Luke, We are using Kafka 2.8.1 Broker/Client system in our prod environment with SASL_SSL communication between Kafka Clients and Broker. We are using the IP for the property "bootstrap.servers" while initiating the KafkaConsumer. Due to some reason, one of our Customer is unable to use the IP in the CA certificate and provided only hostname in the SAN entry in the certificate due to which he is getting following exception in the logs: org.a...

Hello, Since you've installed the Kafka broker you'll need an extra component to get data to or from the database. You can use Kafka Connect https://kafka.apache.org/documentation/#connect with a Debezium CDC Connector https://debezium.io/documentation/reference/1.9/connectors/mysql.html or a JDBC adapter https://github.com/aiven/jdbc-connector-for-apache-kafka . These are just a couple of examples of connectors you can use, there are quite a lot of connectors available. Kind regards, Richard Bosch On Fri, Jul 8, 2022 at 8:57 AM Alalasundaram Saravanan <sara@softintelitech.com.invalid> wrote: > Dear Team, > > We wanted to use kafka and therefore installed on ubuntu ( Ubuntu > 20.04.4 LTS ) kafka ( 3.2.0 & zoopkeeper > 3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a) . > > > With this installation, we wanted to connect to mysql server installed > on the same server. > > Is it possible? If so what d...

Dear Team, We wanted to use kafka and therefore installed on ubuntu ( Ubuntu 20.04.4 LTS ) kafka ( 3.2.0  & zoopkeeper 3.6.3--6401e4ad2087061bc6b9f80dec2d69f2e3c8660a) . With this installation, we wanted to connect to mysql server installed on the same  server. Is it possible? If so what document to follow ? Or We need to install and additional like confluent etc .. With warm regards Alalasundaram Saravanan

Hi Luke, We are using Kafka 2.8.1 Broker/Client system in our prod environment with SASL_SSL communication between Kafka Clients and Broker. We are using the IP for the property "bootstrap.servers" while initiating the KafkaConsumer. Due to some reason, one of our Customer is unable to use the IP in the CA certificate and provided only hostname in the SAN entry in the certificate due to which he is getting following exception in the logs: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed Caused by: javax.net.ssl.SSLHandshakeException: No subject alternative names matching IP address xx.xx.xx.xx found at sun.security.ssl.Alert.createSSLException(Alert.java:131) at sun.security.ssl.TransportContext.fatal(TransportContext.java:324) at sun.security.ssl.TransportContext.fatal(TransportContext.java:267) at sun.security.ssl.TransportContext.fatal(TransportContext.java:262) at sun.security.ssl.CertificateMessage$T12Certif...

Hello kafka-users, I have 50 topics, each with 32 partitions where data is being ingested continuously. Data is being published in these 50 partitions externally (no control) which causes data skew amount the partitions of each topic. For example: For topic-1, partition-1 contains 100 events, while partition-2 can have 10K events and so on for all 50 topics. *Consuming data from all 50 topics using kafka-stream mechanism,* - Running 4 consumer instances, all within the same consumer-group. - Num of threads per consumer process: 8 As data among partitions are not evenly distributed (Data-skewed partitions across topics), I see 1 or 2 consumer instances (JVM) are processing/consuming very less records compared to other 2 instances, My guess is these instances process partitions with less data. *Can someone help, how can I balance the consumers here (distribute consumer workload evenly across 4 consumer instances)? Expectation here is that all 4 consu...