Skip to main content

Posts

Showing posts from June, 2020

你好,我发现了你们kafka-API文档的文本错误问题.

你好,我发现了你们的API文档上的一个错误信息,提供给你们修改,谢谢. 错误问题是 :   KStreamBuilder这个类,并没有 from的方法 . 我这里展示的是kafka 10.0版本..,但愿能修改过来,以免误导开发者们的使用. 谢谢 这是问题网址:  http://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/streams/KafkaStreams.html  

Announcing ApacheCon @Home 2020

Hi, Apache enthusiast! (You're receiving this because you're subscribed to one or more dev or user mailing lists for an Apache Software Foundation project.) The ApacheCon Planners and the Apache Software Foundation are pleased to announce that ApacheCon @Home will be held online, September 29th through October 1st, 2020. We'll be featuring content from dozens of our projects, as well as content about community, how Apache works, business models around Apache software, the legal aspects of open source, and many other topics. Full details about the event, and registration, is available at https://apachecon.com/acah2020 Due to the confusion around how and where this event was going to be held, and in order to open up to presenters from around the world who may previously have been unable or unwilling to travel, we've reopened the Call For Presentations until July 13th. Submit your talks today at https://acna2020.jamhosted.net/ We hope...

How does kafka broker treat log.message.format.version if -lVx is not specified?

Hello . I have a kafka broker 2.4.0 running with below message format setting both in broker and topic. # Broker configuration inter.broker.protocol.version = 2.4-IV1 log.message.format.version = 2.4 # Topic configuration message.format.version=2.4-IV1 I just checked the documentation and realized log.message.format requires -IVX after minor version. https://kafka.apache.org/documentation/#log.message.format.version How does a broker treat log.message.format.version if "-IVX" is not specified? I ended up with below code but looks like I am not looking at the right place... https://github.com/apache/kafka/blob/7f90a58b69cd0eb63ba122b41e6ef6195b0a5d98/core/src/main/scala/kafka/server/KafkaConfig.scala#L126 Thanks, Yu -- Yu Watanabe linkedin: www.linkedin.com/in/yuwatanabe1/ twitter: twitter.com/yuwtennis

Re: Kafka: Messages disappearing from topics, largestTime=0

Hi there, I'm joining the party a little late on this one, but this is something I encountered at work and I think I can shed some light on the problem at hand. I filed a bug report https://issues.apache.org/jira/browse/KAFKA-10207 and also submitted a pull request https://github.com/apache/kafka/pull/8936 that should resolve the issue. From my investigation, it appears the issue was related to the jvm version we were using and only happened against a zfs mount. We tried ext4 and btrfs successfully under this configuration, but eventually upgraded our jvm and the issue with using zfs disappeared. I hope this helps! On 2020/04/29 12:09:28, Liam Clarke-Hutchinson < l...@adscale.co.nz > wrote: > Hmm, how are you doing your rolling deploys?> > > I'm wondering if the time indexes are being corrupted by unclean> > shutdowns. I've> > been reading code and the only path I could find that led to a largest> > timestamp of 0 wa...

Re: Is SinkContext thread safe?

Ben, My understanding is that you don't need to worry about any thread synchronization. Each task has their own instance of the `SinkTaskContext` and given Kafka Connect's behavior of spreading the tasks over the cluster -- by definition it won't be the same instance. This means that even if you try to lock onto any object -- it won't take effect because each task is running in their own JVM/ClassLoader. Moreover, each SinkTaskContext instance has a internal reference to a Kafka consumer and any requests to its `pause()` method will delegate the execution to the `pause()` method of this Kafka consumer. Finally and as you may know, Kafka make sure that a given partition is read by only one consumer group at a time. Thanks, -- Ricardo On 6/23/20 4:38 PM, Ben Zhang wrote: > Hi, I had read the doc and tried to google it but could find relavent questions/answers. I am curious that is SinkContext thread safe? Say I want to use the SinkConte...

Re: New Website Layout

Looks great!! A small comment about menus: `Get Started` and `Docs` pages have different UX. While Get Started pages use the whole page, Docs have a menu on the left side. I'd like Docs pages to also use most of the page. For instance, config and metrics tables could look more readable using more space. Would it be possible to make the left menu expand/collapse, similar to current Confluence wiki menu? Thanks, Jorge. On Fri, Jun 26, 2020 at 11:49 AM Ben Stopford < ben@confluent.io > wrote: > Hey folks > > We've made some updates to the website's look and feel. There is a staged > version in the link below. > > https://ec2-13-57-18-236.us-west-1.compute.amazonaws.com/ > username: kafka > password: streaming > > Comments welcomed. > > Ben >

New Website Layout

Hey folks We've made some updates to the website's look and feel. There is a staged version in the link below. https://ec2-13-57-18-236.us-west-1.compute.amazonaws.com/ username: kafka password: streaming Comments welcomed. Ben

回复: Re:Re: [ANNOUNCE] New committer: Xi Hu

Thank you, everyone. It is my great honor to be a part of the community. Will make a greater contribution in the coming days. ________________________________ 发件人: Roc Marshal < flinker@126.com > 发送时间: 2020年6月25日 10:20 收件人: users@kafka.apache.org < users@kafka.apache.org > 主题: Re:Re: [ANNOUNCE] New committer: Xi Hu Congratulations ! Xi Hu. Best, Roc Marshal. At 2020-06-25 01:30:33, "Boyang Chen" < reluctanthero104@gmail.com > wrote: >Congratulations Xi! Well deserved. > >On Wed, Jun 24, 2020 at 10:10 AM AJ Chen < ajchen@web2express.org > wrote: > >> Congratulations, Xi. >> -aj >> >> >> >> On Wed, Jun 24, 2020 at 9:27 AM Guozhang Wang < wangguoz@gmail.com > wrote: >> >> > The PMC for Apache Kafka has invited Xi Hu as a committer and we are >> > pleased to announce that he has accepted! >> > >> > Xi ...

Re: _consumer_offsets topic produce rate is dam high

[1593007846456,personas_segment_realtime_flighttransactions,7]::OffsetAndMetadata(offset=41424, leaderEpoch=Optional.empty, metadata=, commitTimestamp=1593060016543, expireTimestamp=None) [1593007846456,personas_segment_realtime_flighttransactions,19]::OffsetAndMetadata(offset=44612, leaderEpoch=Optional.empty, metadata=, commitTimestamp=1593060016543, expireTimestamp=None) [1593007846456,personas_segment_realtime_flighttransactions,32]::OffsetAndMetadata(offset=42204, leaderEpoch=Optional.empty, metadata=, commitTimestamp=1593060016543, expireTimestamp=None) [1593007846456,personas_segment_realtime_flighttransactions,29]::OffsetAndMetadata(offset=42288, leaderEpoch=Optional.empty, metadata=, commitTimestamp=1593060016543, expireTimestamp=None) [1593007846456,personas_segment_realtime_flighttransactions,40]::OffsetAndMetadata(offset=40278, leaderEpoch=Optional.empty, metadata=, commitTimestamp=1593060016543, expireTimestamp=None) [1593007846456,personas_segment_realtime_flighttransacti...