Kafka

Hello All, Has anyone here tried using librdkafka on Linux? I could generate the librdkafka so and .a files and the rdkafka header file as well. But when I integrate it into my project, I see build failures. From the errors, it looks like it is trying to internally build winsock2.h. It almost appears as if there is some internal reference to Win32 even for Linux. Please let me know. Thanks, Anand

Hi – thanks to Bruno Cadonna for this reply (which I seem to have lost in my email system sorry!) but copying from archive here for reference: classic is NOT the pre-2.4 protocol but the pre-KIP-848 protocol. If you use classic and a cooperative assignor like CooperativeStickyAssignor you use KIP-429. Also in 4.0. But not sure about this response, which I unfortunately can't read as I'm linguistically challenged (Australian English only sorry)! Can anyone shed any light? Paul From: 咑慥沬唻 <309001141@qq.com.INVALID> Date: Friday, 4 July 2025 at 6:23 pm To: users < users@kafka.apache.org >, Kafka Users < users@kafka.apache.org >, dev < dev@kafka.apache.org > Subject: 回复：KIP-429 vs. KIP 848? [You don't often get email from 309001141@qq.com.invalid. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] EXTERNAL EMAIL - USE CAUTION when clicking links or attachments 我在那里设置，不接受这样的邮件 &nbsp; 咑慥沬...

我在那里设置，不接受这样的邮件 &nbsp; 咑慥沬唻 309001141@qq.com &nbsp; ------------------&nbsp;原始邮件&nbsp;------------------ 发件人: "Brebner, Paul"<Paul.Brebner@netapp.com.INVALID&gt;; 发送时间: 2025年6月17日(星期二) 上午10:31 收件人: "users"< users@kafka.apache.org &gt;; "dev"< dev@kafka.apache.org &gt;; 主题: KIP-429 vs. KIP 848? 我对从KIP-429的转换感到困惑。 https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol 至KIP-848 https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol 429出现在卡夫卡4.0中，848出现在3.7中（第一次提到group.protocol消费者配置为经典或消费者——名字有点混乱——我假设经典是2.4前的再平衡协议，而消费者是新的协议，实际上由代理管理。😉 ) 但是4.0中你还能用429吗？怎么？我想在4.0中的两个增量再平衡KIP之间做一些性能测试。 致以问候，保罗

Keen to following this one. I'm busy with a snmp source connector... might be interested in talking, pls ping me at georgelza@gmail.com G On Thu, Jul 3, 2025 at 7:13 AM Prateek Kohli <prateek.kohli@ericsson.com.invalid> wrote: > Hi All, > > I am planning to use SNMP trap receiver connecto,r which acts as a server > connector for receiving SNMP traps. > https://github.com/jcustenborder/kafka-connect-snmp > > Since this is a source connector that listens for incoming SNMP messages > (not polling), I'm trying to understand how best to scale it? > > Has anyone used this connector in production? > > * Is it possible to run multiple instances in parallel? > * Any suggestions to improve scalability or reliability for such a > setup? > > Appreciate any inputs or lessons learned. > > Regards, > Prateek Kohli > -- You have the obligation to inform one honestly of the r...

Hi All, I am planning to use SNMP trap receiver connecto,r which acts as a server connector for receiving SNMP traps. https://github.com/jcustenborder/kafka-connect-snmp Since this is a source connector that listens for incoming SNMP messages (not polling), I'm trying to understand how best to scale it? Has anyone used this connector in production? * Is it possible to run multiple instances in parallel? * Any suggestions to improve scalability or reliability for such a setup? Appreciate any inputs or lessons learned. Regards, Prateek Kohli

Hi Ranganath, If messages are only received when a specific partition is assigned, but not when subscribing via a consumer group. This is because: --> The consumer config has enable.auto.commit=false, but no manual offset commits are being made (commitSync() is missing). As a result, Kafka thinks there are no new messages to consume for the group. --> Also, if offsets were already committed earlier, --from-beginning has no effect unless the offsets are reset. *Recommended fixes:* 1. Add kafkaConsumer.commitSync() after polling records in Java code. 2. temporarily set enable.auto.commit=true to allow auto commits. 3. For CLI, reset the group offset using: kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group console --topic input --reset-offsets --to-earliest --execute Regards, Sisindri M. On Thu, Jun 26, 2025 at 1:03 AM Samudrala, Ranganath [USA] <Samudrala_Ranganath@bah.com.invalid> wro...

Hi Sachin, Please check KAFKA-19359 < https://issues.apache.org/jira/browse/KAFKA-19359 > for more info. Thanks. Luke On Thu, Jun 26, 2025 at 5:44 PM Sachin Jangle <sachin.jangle@oracle.com.invalid> wrote: > Hi, > > A CVE-2025-48734, has been identified in the third-party library > commons-beanutils version 1.9.4. > Requesting confirmation on the following: > > * Is a fix available in a later version of kafka4 ? > > * If not, is there any recommended workaround or mitigation for the > current version? > Thanks, > Sachin Jangle >