Skip to main content

Re: [DISCUSS] Custom Metadata on Topics — Inconsistent Behavior Between Broker and Topic Config in Kafka 4.2.0

hi Are you looking for an official way to add "labels" to topics? If so, there is no such mechanism for now. One workaround is to create a compacted topic to store them. Another is to encode the labels within the topic name, such as a-b-c-d. If topic-level labels are a crucial feature for you, please feel free to open a JIRA and KIP for it. Best, Chia-Ping On 2026/04/13 03:07:15 Satyaprakash Dwibedi via users wrote: > Hello Kafka Community, > > We have a use case that requires attaching custom metadata to topics — > things like SLA, durability guarantees, and PagerDuty account references. > While exploring this in Kafka 4.2.0, we noticed an interesting > inconsistency between broker-level and topic-level configuration behavior. > > Broker-level custom configs work fine: > > Running the following command succeeds without errors: > > ./bin/kafka-configs.sh --bootstrap-server localhost:9092 \ > --entity-type brokers --entity-name 1 \ > --alter --add-config 'durability.level=1' > > And describing the broker returns the config as expected: > > ./bin/kafka-configs.sh --bootstrap-server localhost:9092 \ > --entity-type brokers --entity-name 1 \ > --describe > > Output: > > Dynamic configs for broker 1 are: > durability.level=null sensitive=true > synonyms={DYNAMIC_BROKER_CONFIG:durability.level=null} > > Topic-level custom configs throw an exception: > > Running a similar command on a topic: > > ./bin/kafka-configs.sh --bootstrap-server localhost:9092 \ > --entity-type topics --entity-name test-topic-metadata \ > --alter --add-config 'durability.level=3' > > ...results in an InvalidConfigurationException. > > Our question: > > Is this asymmetry intentional? Is there a design reason why unknown/custom > configurations are permitted at the broker level but rejected at the topic > level? And is there a recommended approach for attaching arbitrary custom > metadata to topics in Kafka? > > Any guidance or pointers to relevant KIPs would be greatly appreciated! > > Thanks in advance [image: :pray:] >

Comments