Kafka

Hi Avishek, Unfortunately, disabling tiered storage only works under KRaft mode. If you look closely at the KIP-950[1], you'll see this sentence (see the underlined section): > *Support for Both KRaft and Zookeeper-backed Clusters*: The tiered storage disablement feature should be designed and implemented to support both KRaft and Zookeeper-backed Kafka clusters, ensuring compatibility across different cluster configurations. Updated: *We will implement for KRaft only in v3.9.0.* I agree that the KIP itself seems to state that it will work with both. But in the end it was only implemented in KRaft mode. Also, one can see in the documentation PR[2] that it only mentions how to disable it in KRaft mode. Sorry this is not the answer you were looking for. [1]: https://cwiki.apache.org/confluence/display/KAFKA/KIP-950%3A++Tiered+Storage+Disablement [2]: https://github.com/apache/kafka/pull/16836/changes On Wed, Feb 18, 2026 at 1:53 PM Avishek Das via users...

Hello Jian, Thanks for sharing the references. I just wanted to confirm, did you encounter the same issue where the RLMCopyTask was created but did not copy segments to remote storage, or was the RLMCopyTask not created at all in your case? On Tue, Feb 17, 2026 at 8:48 AM jian fu < fujian1115@gmail.com > wrote: > Hi Avishek Das: > > Though I am not sure about your issue. but I encounter the similar issue > and fix it with: > > https://github.com/apache/kafka/pull/20007 > https://github.com/apache/kafka/pull/20203 > https://cwiki.apache.org/confluence/x/Hg9JFg > > You can refer to it to see if it is the same issue. thanks > > Regards > Jian > > Avishek Das via users < users@kafka.apache.org > 于2026年2月16日周一 21:32写道： > > > Hello Team, > > > > During tiered storage performance testing on Kafka 3.9.1, we observed > cases > > where a broker did not trigger scheduled RL...

Hello Community, As part of our testing, I am validating the scenario where tiered storage is disabled on a topic where it was previously enabled. I attempted to alter the topic configuration using the following configs: remote.storage.enable=true remote.log.copy.disable=true However, this results in an exception stating that remote.log.copy.disable is not valid in ZooKeeper mode. Caused by: org.apache.kafka.common.errors.InvalidConfigurationException: It is invalid to set `remote.log.delete.on.disable` or `remote.log.copy.disable` under Zookeeper's mode. We are running Kafka v3.9.1 on a ZooKeeper-backed cluster, and based on the KIP-950 < https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424654#KIP950:TieredStorageDisablement-StateTransitions > documentation, these configurations appear to be available for ZooKeeper-based clusters as well. Am I missing any prerequisite or additional step required to disable Tiered Storage ...

The Apache Kafka community is pleased to announce the release for Apache Kafka 4.2.0 This release has many exciting changes: * Kafka Queues (Share Groups) is now production-ready with new features like the RENEW acknowledgement type for extended processing times, adaptive batching for share coordinators, soft and strict enforcements of quantity of fetched records, and comprehensive lag metrics. * Kafka Streams brings the server-side rebalance protocol to GA with a limited feature set, adds dead letter queue support in exception handlers, introduces anchored wall-clock punctuation for deterministic scheduling, and gives users full control over whether to send a leave group request on closing. * This release also delivers significant improvements to consistency and observability: CLI tools now feature standardized arguments like –bootstrap-server across all tools, metric naming has been corrected to follow the kafka.COMPONENT convention, and new idle ratio metrics pr...

Hi Avishek Das: Though I am not sure about your issue. but I encounter the similar issue and fix it with: https://github.com/apache/kafka/pull/20007 https://github.com/apache/kafka/pull/20203 https://cwiki.apache.org/confluence/x/Hg9JFg You can refer to it to see if it is the same issue. thanks Regards Jian Avishek Das via users < users@kafka.apache.org > 于2026年2月16日周一 21:32写道： > Hello Team, > > During tiered storage performance testing on Kafka 3.9.1, we observed cases > where a broker did not trigger scheduled RLMCopy tasks, resulting in no > segment uploads to remote storage. Scenarios observed: > > - During a rolling restart while tiered storage was actively copying > segments, one broker (out of six) stopped copying segments after > restart. > > - After multiple rolling restarts before enabling tiered storage at > topic level, enabling tiered storage again caused one broker (out of > six...

Hi Anand, The partitionsFor API does not guarantee that a single call will return the data if the topic does not exist. It issues a first metadata request to create the topic, and waits for the api timeout for a response (timeout param or default.api.timeout.ms config). Then returns whatever it gets from the broker in that single call (empty if the topic is not found within the timeout, described in the java docs https://github.com/apache/kafka/blob/e678b4bb7ca99c7d1be0d554ffe7e66f584771d6/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L1415 ). Also there are old integration tests with the pattern you describe: a first call to ensure topic is created, then a following one expected to retrieve the partitions for the topic (e.g, in 3.9 branch https://github.com/apache/kafka/blob/5e9866f43ab8e7e41ef39e5584ac50019381328d/core/src/test/scala/integration/kafka/api/PlaintextConsumerTest.scala#L176 ) That being said, I wonder if the difference you...

Hello Team, During tiered storage performance testing on Kafka 3.9.1, we observed cases where a broker did not trigger scheduled RLMCopy tasks, resulting in no segment uploads to remote storage. Scenarios observed: - During a rolling restart while tiered storage was actively copying segments, one broker (out of six) stopped copying segments after restart. - After multiple rolling restarts before enabling tiered storage at topic level, enabling tiered storage again caused one broker (out of six) to stop copying segments. In both cases, restarting the affected broker resolved the issue and copying resumed. DEBUG logging showed that RLMCopy tasks were created for leader partitions, traffic was active, and rolled segments were available. However, no debug logs appeared from the actual copy path < https://github.com/apache/kafka/blob/3.9/core/src/main/java/kafka/log/remote/RemoteLogManager.java#L839 >, suggesting the copy workflow was not being tri...