Skip to main content

Posts

Re: What does Apache Kafka do?

Hi Good evening from India😃 I am using Kafka(confluent Platform) since long time. Its actually a message broker in simple term which can persist data on disk for short time in days or weeks. It acts like a buffer in between message producer and message consumer. Kafka reduces loading of consumers by acting as a buffer in real time data processing apps. In short.. you can achieve lose coupling between producers and consumers using kafka. (Asynchronous communication) And there are lot of other features like Rest Proxy, connectors, Schema Registry etc. you need to invest time in reading for good returns in long term 😀 I hope it helps. Cheers, Sunil. On Wed, 18 May 2022 at 8:01 PM, Turritopsis Dohrnii Teo En Ming < ceo.teo.en.ming@gmail.com > wrote: > Subject: What does Apache Kafka do? > > Good day from Singapore, > > I notice my company/organization is using Apache Kafka. What does it do? > > Just being curious. > >...

What does Apache Kafka do?

Subject: What does Apache Kafka do? Good day from Singapore, I notice my company/organization is using Apache Kafka. What does it do? Just being curious. Regards, Mr. Turritopsis Dohrnii Teo En Ming Targeted Individual in Singapore 18 May 2022 Wed

Re: [ANNOUNCE] Apache Kafka 3.2.0

Thanks for running the release Bruno! On Tue, May 17, 2022 at 12:02 PM Bruno Cadonna < cadonna@apache.org > wrote: > The Apache Kafka community is pleased to announce the release for Apache > Kafka 3.2.0 > > * log4j 1.x is replaced with reload4j (KAFKA-9366) > * StandardAuthorizer for KRaft (KIP-801) > * Send a hint to the partition leader to recover the partition (KIP-704) > * Top-level error code field in DescribeLogDirsResponse (KIP-784) > * kafka-console-producer writes headers and null values (KIP-798 and > KIP-810) > * JoinGroupRequest and LeaveGroupRequest have a reason attached (KIP-800) > * Static membership protocol lets the leader skip assignment (KIP-814) > * Rack-aware standby task assignment in Kafka Streams (KIP-708) > * Interactive Query v2 (KIP-796, KIP-805, and KIP-806) > * Connect APIs list all connector plugins and retrieve their > configuration (KIP-769) > * TimestampConverter SMT supports d...

[ANNOUNCE] Apache Kafka 3.2.0

The Apache Kafka community is pleased to announce the release for Apache Kafka 3.2.0 * log4j 1.x is replaced with reload4j (KAFKA-9366) * StandardAuthorizer for KRaft (KIP-801) * Send a hint to the partition leader to recover the partition (KIP-704) * Top-level error code field in DescribeLogDirsResponse (KIP-784) * kafka-console-producer writes headers and null values (KIP-798 and KIP-810) * JoinGroupRequest and LeaveGroupRequest have a reason attached (KIP-800) * Static membership protocol lets the leader skip assignment (KIP-814) * Rack-aware standby task assignment in Kafka Streams (KIP-708) * Interactive Query v2 (KIP-796, KIP-805, and KIP-806) * Connect APIs list all connector plugins and retrieve their configuration (KIP-769) * TimestampConverter SMT supports different unix time precisions (KIP-808) * Connect source tasks handle producer exceptions (KIP-779) All of the changes in this release can be found in the release notes: https://www.apache.org/dist/...

kafka stream - sliding window - getting unexpected output

Hi All, Our use case is to use sliding window. (for e.g. at any point, whenever > user performs any actions at time [ t1 ], we would like to see his activity > in [ t1 - last 24 hours]. Using this, to show the user some recommendations. -- I have code ready and it works without any errors. -- aggregations happen as expected. -- but the output generated is unexpected. As windows gets slides, i am getting mixed output which includes intermediate aggregated records also coming with final aggregated outputs. Could someone please help me here ? what can I do here to get ONLY final aggregated output. Code snippet : ________________________________________________________________ builder.stream(tempReadingTopic, Consumed.with(stringSerde, inputSerde)) .filter((k, v) -> v != null) .map((k,v) -> KeyValue.pair(v.getUserId(), v)) //.through("slidingbykey", Produced.with(Serdes.String(), inp...

AW: Switchin from Zookepper to Kafka KRaft mode / Using ACLs with Kafka KRaft mode

Hi Thomas, thanks for the quick answer. I got it running by declaring the CONTROLLER listerner also as SSL and using the existing keystore/trustore for it. Unfortunately I now recognize a new issue when restarting Kafka: https://issues.apache.org/jira/browse/KAFKA-13909 Hopefully this will also be solved. Awaiting using Kafka without ZooKeeper. Best regards, Florian -----Ursprüngliche Nachricht----- Von: Thomas Cooper [mailto: code@tomcooper.dev ] Gesendet: Montag, 16. Mai 2022 14:49 An: users@kafka.apache.org ; Florian Blumenstein < florian.blumenstein@bbraun.com > Betreff: Re: Switchin from Zookepper to Kafka KRaft mode / Using ACLs with Kafka KRaft mode CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi Florian, Switching from a Zookeeper based cluster to a KRaft based one is not currently supported. AFAIK that functionality sh...

Re: [kafka-clients] Re: [VOTE] 3.2.0 RC1

Hi Bruno, +1 (non binding) - Checked documentation and blog post - Ran some client tests using the staged artifacts - Verified the signing key and sha512 sum - Built from source - Ran unit and integration tests Thanks Fede On Thu, May 12, 2022 at 12:03 PM Bruno Cadonna < cadonna@apache.org > wrote: > > Hi all, > > Please review the blog post for the Apache Kafka 3.2.0 release: > > https://blogs.apache.org/preview/kafka/?previewEntry=what-s-new-in-apache8 > > I will accept comments until Monday, May 16th EOD PT. > > Best, > Bruno > > > On 06.05.22 14:52, 'David Jacot' via kafka-clients wrote: > > Thanks for running the release, Bruno. > > > > I performed the following validations: > > * Verified all checksums and signatures. > > * Built from source and ran unit tests. > > * Ran the first quickstart steps for both ZK and KRaft. > > * Spotchecked the doc...