Skip to main content

Posts

Showing posts from July, 2023

RE: Kafka Connect Rest Extension Question

Hi Team, Greetings, We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC IT SERVICE PARTNERSHIP" "IT SERVICE OUTSOURCING" " "PARTNER SERVICE SUBCONTRACTING" We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White Label Model. Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee payroll, Help partner to get profit more than 50% on each project.. ..We really mean it. We are already working with platinum partner like NTT DATA, NEC Singapore, Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc. Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB kailash@votecgroup.com | Partnership In charge | Let us know your availability this week OR Next week?? -----Original Message----- From: miltan@votecgroup.com [mailto: miltan@votecgroup.com ] Sent: 01 August 2023 ...

RE: Kafka Connect Rest Extension Question

Hi Team, Greetings, We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC IT SERVICE PARTNERSHIP" "IT SERVICE OUTSOURCING" " "PARTNER SERVICE SUBCONTRACTING" We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White Label Model. Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee payroll, Help partner to get profit more than 50% on each project.. ..We really mean it. We are already working with platinum partner like NTT DATA, NEC Singapore, Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc. Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB kailash@votecgroup.com | Partnership In charge | Let us know your availability this week OR Next week?? -----Original Message----- From: ģ–‘ķ˜•ģš± [mailto: hyungwookyang@navercorp.com ] Sent: 31 July 2023 14:42 To: user...

RE: Kafka Connect Rest Extension Question

Hi Team, Greetings, We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC IT SERVICE PARTNERSHIP" "IT SERVICE OUTSOURCING" " "PARTNER SERVICE SUBCONTRACTING" We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White Label Model. Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee payroll, Help partner to get profit more than 50% on each project.. ..We really mean it. We are already working with platinum partner like NTT DATA, NEC Singapore, Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc. Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB kailash@votecgroup.com | Partnership In charge | Let us know your availability this week OR Next week?? -----Original Message----- From: Greg Harris [mailto: greg.harris@aiven.io.INVALID ] Sent: 31 July 2023 23:42 ...

Kafka User Authentication Exception

Dear Kafka Community Team, I am a developer who is facing an issue while attempting to use SASL authentication with Kafka. I have tried various methods to troubleshoot the problem, but have not yet found a solution. Therefore, I am reaching out to seek your help and guidance. Issue Description: I am trying to connect to Kafka from my client application using the SASL authentication mechanism. In my client code, I have implemented the `SaslClientCallbackHandler` class to handle callbacks and provided the correct credentials (username and password). However, during the connection process, I encounter the "javax.security.sasl.SaslException: User name could not be obtained" error. I have attempted the following steps to troubleshoot the issue: 1. Ensured that the client and Kafka Broker are using the same version of Kafka to avoid version compatibility issues. 2. Checked the SASL configuration to ensure that both the client and Kafka Broker are configured correctly with the same ...

Re: Kafka Connect Rest Extension Question

Hello Yang Hyung Wook, In your post I do not see anything obviously wrong, so you may need to do some more debugging. 1. Are you using the same jar for both the classpath and plugin.path tests? If not, do they both contain the service loader manifest file? You can test this with https://docs.oracle.com/javase/tutorial/deployment/jar/view.html 2. Do you see either of these errors https://github.com/apache/kafka/blob/3.5.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/DelegatingClassLoader.java#L267-L269 for filesystem-specific problems? 3. This log line https://github.com/apache/kafka/blob/3.5.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/isolation/DelegatingClassLoader.java#L275 should be printed multiple times when you're using plugin.path. Are you seeing a log including a directory containing your jar file, the jar file itself, or only other locations? 4. If there is a dependency-related error, it will appear wit...

Kafka Connect Rest Extension Question

https://stackoverflow.com/questions/76797743/how-can-i-solve-connectrestextension-error There is an issue with this link where the ConnectRestExtension implementation is not registered. I've done everything the kip official documentation says, but can you tell me why it doesn't work? ģ–‘ķ˜•ģš± Yang Hyung Wook Global Platform Dev ź²½źø°ė„ ģ„±ė‚Øģ‹œ 분당구 ė¶„ė‹¹ė‚“ź³”ė”œ 131 ķŒźµķ…Œķ¬ģ› ķƒ€ģ›Œ1 (우)13529 Tel Mobile 010-2815-2145 Email hyungwookyang@navercorp.com ģœ„ ģ „ģžģš°ķŽø ė° 그에 ķ¬ķ•Øėœ ģ •ė³“ėŠ” ģœ„ģ— źø°ģž¬ėœ ģˆ˜ģ‹ ģøė§Œģ„ ģœ„ķ•“ ė°œģ†”ė˜ėŠ” ź²ƒģœ¼ė”œģ„œ ė³“ģ•ˆģ„ ģœ ģ§€ķ•“ģ•¼ ķ•˜ėŠ” 정볓 ė° ė²•ė„ ģƒ ė˜ėŠ” 다넸 ģ‚¬ģœ ė”œ ģøķ•˜ģ—¬ ź³µź°œź°€ źøˆģ§€ėœ 정볓가 들얓 ģžˆģ„ 수 ģžˆģŠµė‹ˆė‹¤. ź·€ķ•˜ź°€ ģ“ ģ „ģžģš°ķŽøģ˜ 지정 ģˆ˜ģ‹ ģøģ“ ģ•„ė‹ˆė©“ ģ“ė„¼ ė¬“ė‹Øģœ¼ė”œ 볓유, 복제, 전솔, ė°°ķ¬, ź³µź°œķ•  수 ģ—†ģœ¼ė©°, ģ¼ė¶€ģ˜ ė‚“ģš©ģ“ė¼ė„ 볓유, 복제, ė°°ķ¬, ź³µź°œķ•“ģ„œėŠ” ģ•ˆė©ė‹ˆė‹¤. ź·øėŸ¬ėÆ€ė”œ, ģž˜ėŖ» ģˆ˜ģ‹ ėœ ź²½ģš°ģ—ėŠ” ģ¦‰ģ‹œ ė„¤ģ“ė²„ ķ“ė¼ģš°ė“œ ź°œģøģ •ė³“ė³“ķ˜ø( dl_ncloud_privacy@navercorp.com )딜 ģ—°ė½ķ•˜ģ—¬ ģ£¼ģ‹œź³ , 원본 ė° 사본과 그에 따넸 첨부 ė¬øģ„œė„¼ 모두 ģ‚­ģ œķ•˜ģ—¬ ģ£¼ģ‹œźø° ė°”ėžė‹ˆė‹¤. ķ˜‘ģ”°ķ•˜ģ—¬ ģ£¼ģ…”ģ„œ ź°ģ‚¬ķ•©ė‹ˆė‹¤. ​ This email and the information contained in this email are intended solely for the recipient(s) addressed above and may contain information that is confidential and/or privileged or whose disclosure is prohibited by law or ot...

KRaft controller listener with SASL SCRAM-SHA-512

Hi, I've been trying to get SASL using SCRAM-SHA-512 up and running in Kafka 3.5 but I've been running into some issues which I'm not sure how to debug. I'm able to authenticate when I use PLAIN and the following kafka_jaas.conf: KafkaServer { org.apache.kafka.common.security.plain.PlainLoginModule required username="mybrokeruser" password="mybrokerpassword"; }; I then try to change sasl.mechanism.controller.protocol from PLAIN to SCRAM-SHA-512, run kafka-storage.sh format with --add-scram and change my kafka_jaas.conf: KafkaServer { org.apache.kafka.common.security.scram.ScramLoginModule required username="mybrokeruser" password="mybrokerpassword"; }; At that point I keep getting authentication errors and I'm not sure how to debug it or find out which credentials are used. I tried setting all loggers to TRACE but I wasn't able to figure it out. It's working on ...

Re: Unable to launch KRaft

Hi Adrian, Please follow the quick start guide in Kafka doc: https://kafka.apache.org/quickstart Thanks. Luke On Fri, Jul 28, 2023 at 3:53 PM adrien ruffie < adriennolarsen@hotmail.fr > wrote: > dear all, > > I would like to try KRaft in but I didn't arrived to finish the > installation bellow: > > > https://developer.confluent.io/learn/kraft/?utm_medium=sem&utm_source=google&utm_campaign=ch.sem_br.nonbrand_tp.prs_tgt.dsa_mt.dsa_rgn.emea_lng.eng_dv.all_con.confluent-developer&utm_term=&creative=&device=c&placement=&gad=1&gclid=CjwKCAjwzo2mBhAUEiwAf7wjknGluNtB_3DDul3Qrb5Usv_nybLBLkeoqMnczaAmdhRUk6nGh-WisxoCG2wQAvD_BwE > > > When I try ./bin/kafka-server-start.sh -daemon > ./config/kraft/server.properties nothing happen consequently I tried > without daemon et after that I got the following error: > > > > I tried after to make a kafka-storage.sh but without provid...

Unable to launch KRaft

dear all, I would like to try KRaft in  but I didn't arrived to finish the installation bellow: https://developer.confluent.io/learn/kraft/?utm_medium=sem&utm_source=google&utm_campaign=ch.sem_br.nonbrand_tp.prs_tgt.dsa_mt.dsa_rgn.emea_lng.eng_dv.all_con.confluent-developer&utm_term=&creative=&device=c&placement=&gad=1&gclid=CjwKCAjwzo2mBhAUEiwAf7wjknGluNtB_3DDul3Qrb5Usv_nybLBLkeoqMnczaAmdhRUk6nGh-WisxoCG2wQAvD_BwE When I try  ./bin/kafka-server-start.sh  -daemon ./config/kraft/server.properties     nothing happen consequently I tried without daemon et after that I got the following error: I tried after to make a kafka-storage.sh but without providing any arguments, the raised error was "Too few arguments" ... anyone have a idea how I can launch correctly kafka with KRaft plz ? I'm on an hyperv machin with ubuntu 20.04.2 LTS Thank a lot and...

RE: Streams/RocksDB: Why Universal Compaction?

Hi Team, Greetings, We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC IT SERVICE PARTNERSHIP" "IT SERVICE OUTSOURCING" " "PARTNER SERVICE SUBCONTRACTING" We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White Label Model. Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee payroll, Help partner to get profit more than 50% on each project.. ..We really mean it. We are already working with platinum partner like NTT DATA, NEC Singapore, Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc. Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB kailash@votecgroup.com | Partnership In charge | Let us know your availability this week OR Next week?? -----Original Message----- From: Colt McNealy [mailto: colt@littlehorse.io ] Sent: 26 July 2023 21:08 To: use...

RE: Consuming an entire partition with control messages

Hi Team, Greetings, We actually reached out to you for Oracle/ IT / SAP / Infor / Microsoft "VOTEC IT SERVICE PARTNERSHIP" "IT SERVICE OUTSOURCING" " "PARTNER SERVICE SUBCONTRACTING" We have very attractive newly introduce reasonably price PARTNER IT SERVICE ODC SUBCONTRACTING MODEL in USA, Philippines, India and Singapore etc with White Label Model. Our LOW COST IT SERVICE ODC MODEL eliminate the cost of expensive employee payroll, Help partner to get profit more than 50% on each project.. ..We really mean it. We are already working with platinum partner like NTT DATA, NEC Singapore, Deloitte, Hitachi consulting. ACCENTURE, Abeam Singapore etc. Are u keen to understand VOTEC IT PARTNERSHIP offerings? Looping KB kailash@votecgroup.com | Partnership In charge | Let us know your availability this week OR Next week?? -----Original Message----- From: Matthias J. Sax [mailto: mjsax@apache.org ] Sent: 27 July 2023 22:05 To: use...

Re: Streams/RocksDB: Why Universal Compaction?

Thanks Colt! For a library's default configs, I think the principle would be "it runs appropriately out of the box for the first time you played with it", so I'm not suggesting we should try to make sure it is a generally good combination for a wide range of production usage since in that case most people would prefer to have some customizations rather than blindly accept those default config values anyways. Hence, what I had in mind regarding "benchmarks" is something light, like running those stateful examples in our tutorials ( https://github.com/apache/kafka/tree/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples ), and see if the new config over all gives a better performance. It's not required to be a very comprehensive one. If you could help with that validation it would be great. Guozhang On Wed, Jul 26, 2023 at 8:38 AM Colt McNealy < colt@littlehorse.io > wrote: > > Guozhang, > > Th...

Re: Consuming an entire partition with control messages

Well, `kafka-consumer-group.sh` can only display the difference between "committed offset" and "end offset". It cannot know what the "right" offset to be committed is. It's really the responsibility of the consumers to commit correctly. -Matthias On 7/27/23 1:03 AM, Vincent Maurin wrote: > Thank you Matthias for your answer, I open an issue on the aiokafka > project as follow up, let's see how we can resolve it there > https://github.com/aio-libs/aiokafka/issues/911 > > As mentioned in the issue, some tools like kafka-consumer-groups.sh also > display a lag of "1" in this kind of situation > > Best regards, > > Vincent > > On 13/06/2023 17:27, Matthias J. Sax wrote: >> Sounds like a bug in aiokafka library to me. >> >> If the last message in a topic partition is a tx-marker, the consumer >> should step over it, and report the correct position ...

Re: Consuming an entire partition with control messages

Thank you Matthias for your answer, I open an issue on the aiokafka project as follow up, let's see how we can resolve it there https://github.com/aio-libs/aiokafka/issues/911 As mentioned in the issue, some tools like kafka-consumer-groups.sh also display a lag of "1" in this kind of situation Best regards, Vincent On 13/06/2023 17:27, Matthias J. Sax wrote: > Sounds like a bug in aiokafka library to me. > > If the last message in a topic partition is a tx-marker, the consumer > should step over it, and report the correct position after the marker. > > The official KafkaConsumer (ie, the Java one), does the exact same thing. > > > -Matthias > > On 5/30/23 8:41 AM, Vincent Maurin wrote: >> Hello ! >> >> I am working on an exactly once stream processors in Python, using >> aiokafka client library. My program stores a state in memory, that is >> recovered from a changelog topi...

Re: Streams/RocksDB: Why Universal Compaction?

Guozhang, Thanks for your response. That makes a lot of sense; I can't promise any super-formal benchmarks but we will definitely play with the configurations you sent and report back within a month about our high-level findings. For our purposes (a workflow engine), we will mostly monitor workflow execution metrics + state store restoration times. But in the interest of a formal benchmark that could be included in a KIP—what monitoring software tooling and setup environment would you recommend? If it doesn't involve writing copious amounts of custom code, perhaps (no promises) my team could put something together that's more suitable for a general Streams audience rather than just our own internal usage. Cheers, Colt McNealy *Founder, LittleHorse.dev* On Sun, Jul 23, 2023 at 11:21 AM Guozhang Wang < guozhang.wang.us@gmail.com > wrote: > Yeah I can shed some light here: I used Universal originally since at > the beginning of Kafka ...

Re: Release plan required for version 3.5.1

Hi Sahil, Please have a look at the dependencies for Kafka 3.5.1: https://github.com/apache/kafka/blob/3.5.1/gradle/dependencies.gradle and compare it with your list of CVEs. Please also have a look here: https://kafka.apache.org/project-security If you discover a security issue please follow the instructions on that page and engage to resolve the security issue. Best, Bruno On 7/26/23 6:20 AM, Sahil Sharma D wrote: > Hi Kamal, > > Shall we consider CVEs mentioned in mail trail are fixed in v3.5.1? > > We are unable to find the CVEs in Jira as suggested earlier. > > Regards, > Sahil > > -----Original Message----- > From: Kamal Chandraprakash < kamal.chandraprakash@gmail.com > > Sent: 26 July 2023 09:42 AM > To: users@kafka.apache.org > Subject: Re: Release plan required for version 3.5.1 > > Hi Sahil, > > Apache Kafka 3.5.1 is already released: https://kafka.apache.org/downloads...

RE: Release plan required for version 3.5.1

Hi Kamal, Shall we consider CVEs mentioned in mail trail are fixed in v3.5.1? We are unable to find the CVEs in Jira as suggested earlier. Regards, Sahil -----Original Message----- From: Kamal Chandraprakash < kamal.chandraprakash@gmail.com > Sent: 26 July 2023 09:42 AM To: users@kafka.apache.org Subject: Re: Release plan required for version 3.5.1 Hi Sahil, Apache Kafka 3.5.1 is already released: https://kafka.apache.org/downloads On Wed, Jul 26, 2023 at 9:08 AM Sahil Sharma D <sahil.d.sharma@ericsson.com.invalid> wrote: > Gentle reminder-2 > > -----Original Message----- > From: Sahil Sharma D > Sent: 12 July 2023 09:51 AM > To: users@kafka.apache.org > Subject: RE: Release plan required for version 3.5.1 > > Gentle reminder! > > -----Original Message----- > From: Sahil Sharma D > Sent: 03 July 2023 04:39 PM > To: users@kafka.apache.org > Subject: RE: Release plan required for vers...

Re: Release plan required for version 3.5.1

Hi Sahil, Apache Kafka 3.5.1 is already released: https://kafka.apache.org/downloads On Wed, Jul 26, 2023 at 9:08 AM Sahil Sharma D <sahil.d.sharma@ericsson.com.invalid> wrote: > Gentle reminder-2 > > -----Original Message----- > From: Sahil Sharma D > Sent: 12 July 2023 09:51 AM > To: users@kafka.apache.org > Subject: RE: Release plan required for version 3.5.1 > > Gentle reminder! > > -----Original Message----- > From: Sahil Sharma D > Sent: 03 July 2023 04:39 PM > To: users@kafka.apache.org > Subject: RE: Release plan required for version 3.5.1 > > Hi, > > That means below vulnerabilities are not appliable for kafka, right? > CVE-2022-42003 > CVE-2022-42004 > CVE-2023-34454 > CVE-2023-34453 > CVE-2023-35116 > > Regards, > Sahil > > -----Original Message----- > From: Josep Prat <josep.prat@aiven.io.INVALID> > Sent: 03 July 2023 02:02 PM ...

RE: Release plan required for version 3.5.1

Gentle reminder-2 -----Original Message----- From: Sahil Sharma D Sent: 12 July 2023 09:51 AM To: users@kafka.apache.org Subject: RE: Release plan required for version 3.5.1 Gentle reminder! -----Original Message----- From: Sahil Sharma D Sent: 03 July 2023 04:39 PM To: users@kafka.apache.org Subject: RE: Release plan required for version 3.5.1 Hi, That means below vulnerabilities are not appliable for kafka, right? CVE-2022-42003 CVE-2022-42004 CVE-2023-34454 CVE-2023-34453 CVE-2023-35116 Regards, Sahil -----Original Message----- From: Josep Prat <josep.prat@aiven.io.INVALID> Sent: 03 July 2023 02:02 PM To: users@kafka.apache.org Subject: Re: Release plan required for version 3.5.1 Hi Sahil, Thanks for caring about Apache Kafka's security. One can fix this situation by replacing the affected jar file with the one containing the fix for the vulnerabilities. We plan to add a write up under Apache Kafka's CVE page. Mind that A...

Re: kafka streams re-partitioning on incoming events

Thanks a lot Bruno! I am just trying the Processor API as you mentioned above, so the processor will write record to another kafka topic with new key. I am just having difficulty to read in another processor from that kafka topic and wondering if I need to create another stream with source as intermediate kafka topic? Else how can I read from intermediate topic in another processor? On Fri, Jul 14, 2023 at 9:25 PM Bruno Cadonna < cadonna@apache.org > wrote: > Hi Pushkar, > > The events after repartitioning are processed by a different task than > the task that read the events from the source topic. The task assignor > assigns those tasks to stream threads. So events with the same key will > be processed by the same task. As far as I understood from your earlier > message, you want that events with same call ID after the repartitioning > are processed by the same applications instance. If same keys are > processed by the same task th...

Output broker events to a log file

Classification: Public Hi, I would like to output an event to a log file when certain actions occur on the broker to build up an audit trail for example: Producer connected / disconnected Consumer connected / disconnected Message received Message consumed I do this already on ActiveMQ by using the plugins functionality and hooking into send and receive events etc. Does Kafka provide a similar plugin mechanism that could be extended? Thanks

[ANNOUNCE] Apache Kafka 3.5.1

The Apache Kafka community is pleased to announce the release for Apache Kafka 3.5.1 This release is a security patch release. It upgrades the dependency, snappy-java, to a version which is not vulnerable to CVE-2023-34455. You can find more information about the CVE at Kafka CVE list < https://kafka.apache.org/cve-list#CVE-2023-3445 >. Additionally, this releases fixes a regression introduced in 3.3.0, which caused security.protocol configuration values to be restricted to upper case only. With this release, security.protocol values are case insensitive. See KAFKA-15053 < https://issues.apache.org/jira/browse/KAFKA-15053 > for details. All of the changes in this release can be found in the release notes: https://www.apache.org/dist/kafka/3.5.1/RELEASE_NOTES.html You can download the source and binary release (Scala 2.11 and Scala 2.12) from: https://kafka.apache.org/downloads#3.5.1 ----------------------------------------------------------------...

Re: Streams/RocksDB: Why Universal Compaction?

Yeah I can shed some light here: I used Universal originally since at the beginning of Kafka Streams journey there were user reports complaining about its storage amplifications. But soon enough (around 2019) I've realized that, as a OOTB config, level compaction may be more preferable. I had a PR dating back to that time where I suggested changing a bunch of OOTB configs or RocksDB including the compaction config: https://github.com/apache/kafka/pull/6406/files , unfortunately it was not merged since I wanted to run some benchmarks to make sure it does not have any gotchas but never got the time to do so. I would be very happy in fact if someone could pick that up and re-examine if they still make sense, and if yes drive it through and merge. Guozhang On Sun, Jul 23, 2023 at 10:29 AM Matthias J. Sax < mjsax@apache.org > wrote: > > Do you happen to know? > > > -------- Forwarded Message -------- > Subject: Streams/RocksDB: Why Univ...

Re: Kafka server.log deletion config

Hi Guys, Can someone share any kind of example related to defining policy for server.log deletion in log4j for Kafka Regards, GP On Sat, 22 Jul, 2023, 15:16 Gaurav Pande, < gaupande21@gmail.com > wrote: > Thanks Sunil, could you guide me through any example config for this in > Kafka , any documentation or link that can serve as reference? > Regards, > GP > > On Sat, 22 Jul, 2023, 14:04 sunil chaudhari, < sunilmchaudhari05@gmail.com > > wrote: > >> Hi Gaurav, >> you can make use of log4j.properties for managing log files. >> Define housekeeping policy and get rid of too many files. >> >> >> On Sat, 22 Jul 2023 at 10:58 AM, Gaurav Pande < gaupande21@gmail.com > >> wrote: >> >> > Hi All, >> > >> > I am running Apache kafka 2.7.0 and I see that presently there are many >> > server.logv files but how is the deletion of these file gove...

Re: Kafka server.log deletion config

Thanks Sunil, could you guide me through any example config for this in Kafka , any documentation or link that can serve as reference? Regards, GP On Sat, 22 Jul, 2023, 14:04 sunil chaudhari, < sunilmchaudhari05@gmail.com > wrote: > Hi Gaurav, > you can make use of log4j.properties for managing log files. > Define housekeeping policy and get rid of too many files. > > > On Sat, 22 Jul 2023 at 10:58 AM, Gaurav Pande < gaupande21@gmail.com > > wrote: > > > Hi All, > > > > I am running Apache kafka 2.7.0 and I see that presently there are many > > server.logv files but how is the deletion of these file governed or can > be > > governed? Instead of manually deleting them. > > > > Is there some config that I can set to delete them regularly. > > > > Thanks, > > GP > > >

Re: Kafka server.log deletion config

Hi Gaurav, you can make use of log4j.properties for managing log files. Define housekeeping policy and get rid of too many files. On Sat, 22 Jul 2023 at 10:58 AM, Gaurav Pande < gaupande21@gmail.com > wrote: > Hi All, > > I am running Apache kafka 2.7.0 and I see that presently there are many > server.logv files but how is the deletion of these file governed or can be > governed? Instead of manually deleting them. > > Is there some config that I can set to delete them regularly. > > Thanks, > GP >

Kafka server.log deletion config

Hi All, I am running Apache kafka 2.7.0 and I see that presently there are many server.logv files but how is the deletion of these file governed or can be governed? Instead of manually deleting them. Is there some config that I can set to delete them regularly. Thanks, GP

[RESULTS] [VOTE] Release Kafka version 3.5.1

This vote passes with 4 +1 votes (3 bindings) and no 0 or -1 votes. +1 votes PMC Members: * Mickael Maison * Luke Chen * Chris Egerton Community: * Federico Valeri 0 votes * No votes -1 votes * No votes Vote thread: https://lists.apache.org/thread/jp82w4rw8l24gm5wh0fgnp5370pyq6s8 I'll continue with the release process and the release announcement will follow in the next few days. -- Divij Vaidya Release Manager for Apache Kafka 3.5.1

Re: Kafka data directory Move without data loss

I suggest you practice this thing in some dev or test environment before doing it to prod environment. And copy the data, Not move. On Tue, 18 Jul 2023 at 8:22 PM, sunil chaudhari < sunilmchaudhari05@gmail.com > wrote: > > Better you have a rollback plan. In case the new mountpount has some issue > then you should be able to point to the old fs and restart the broker. > > > On Tue, 18 Jul 2023 at 5:23 PM, Gaurav Pande < gaupande21@gmail.com > wrote: > >> Okay and this process doesn't corrupt the files/file system right? >> >> And to perform this copy activity do we need to stop Kafka broker as well? >> >> Regards, >> GP >> >> On Tue, 18 Jul, 2023, 16:24 sunil chaudhari, < sunilmchaudhari05@gmail.com >> > >> wrote: >> >> > Actually the broker refers to the base directory. >> > Example: >> > log.paths: /var/log/mykafka/...