Re: Explicitly creating topology topics in a streams app

Hi John,

I'm not a Kafka streams expert but have experimented a few times – I recall that Kafka Streams does need to create/use "internal topics" – and security has to be set on clients correctly from memory.
This may help? https://kafka.apache.org/23/documentation/streams/developer-guide/manage-topics
And this
https://kafka.apache.org/23/documentation/streams/developer-guide/security.html#streams-developer-guide-security

Regards, Paul Brebner, NetApp

From: John D. Ament <johndament@apache.org>
Date: Friday, 22 November 2024 at 8:46 am
To: users@kafka.apache.org <users@kafka.apache.org>
Subject: Explicitly creating topology topics in a streams app
[You don't often get email from johndament@apache.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

EXTERNAL EMAIL - USE CAUTION when clicking links or attachments




Hi,

When I look at most stream examples I notice that they tend to include the
broker setting:

auto.create.topics.enable=true

My understanding is that this isn't usually recommended for production
environments, so we have it off. We have started to play with Kafka
Streams apps a bit but noticed that they fail in our production
environments with errors around "UNKNOWN_TOPIC_OR_PARTITION". I suspect
it's related to not having auto create enabled.

Is there an option to force the client to create the topology topics that
they use for Kafka Streams apps? Or is auto creation required?

Thanks,

John