Kafka

Hi list, In https://github.com/apache/kafka/blob/trunk/docker/examples/README.md it says: > If no user provided configuration (file input or environment variables) is passed to the Docker container, the default KRaft configuration for single combined-mode node will be used. This default configuration is packaged in the Kafka tarball. I have two questions about this: - Where is this logic implemented? Is it in kafka.docker.KafkaDockerWrapper? I don't see it. - Can I find this default config outside of the Docker container? It's somewhere in the Git repo I guess? Because I need to "restore" it (by mounting a user config) if I want to set environment variables, correct? Thanks, André

Hi TengYao, I have done the following tests: - Built from 3.9.1-rc1 with OpenJDK-17 and OpenJDK-21. - Ran all unit and integration tests. - Verified basic produce/consume. - Validated the checksums and keys. - Ran KRaft with kafka_2.13-3.9.1.tgz Thanks, PoAn > On Apr 23, 2025, at 5:00 PM, TengYao Chi < frankvicky@apache.org > wrote: > > Hello Kafka users, developers, and client-developers, > > This is the second candidate for the release of Apache Kafka 3.9.1. > This RC fixed the release tag off to a temporary branch issue (KAFKA-19166 > < https://issues.apache.org/jira/browse/KAFKA-19166 >) compared with RC0. > > This is a bug-fix release with several fixes, and most importantly, it adds > Java 23 support for 3.9 > > Release notes for the 3.9.1 release: > https://dist.apache.org/repos/dist/dev/kafka/3.9.1-rc1/RELEASE_NOTES.html > > Please download, test, and vote by *Wednesday, April 30, 9:00...

Hello Kafka users, developers, and client-developers, This is the second candidate for the release of Apache Kafka 3.9.1. This RC fixed the release tag off to a temporary branch issue (KAFKA-19166 < https://issues.apache.org/jira/browse/KAFKA-19166 >) compared with RC0. This is a bug-fix release with several fixes, and most importantly, it adds Java 23 support for 3.9 Release notes for the 3.9.1 release: https://dist.apache.org/repos/dist/dev/kafka/3.9.1-rc1/RELEASE_NOTES.html Please download, test, and vote by *Wednesday, April 30, 9:00 AM PT* Kafka's KEYS file containing PGP keys we use to sign the release: https://kafka.apache.org/KEYS * Release artifacts to be voted upon (source and binary): https://dist.apache.org/repos/dist/dev/kafka/3.9.1-rc1/ * Docker release artifacts to be voted upon: apache/kafka:3.9.1-rc1 https://hub.docker.com/layers/apache/kafka/3.9.1-rc1/images/sha256-459c4e12260a1426a4f53c357d6e0c52b58ed70ff4118de1ffceb3f131d64a...

Hi Jamie, We had success implanting basic authentication using out of the box constructs. Is this what you tried? https://docs.confluent.io/platform/current/security/authentication/http-basic-auth/overview.html We also implemented JWT but we did not have any success doing that out of the box. For that, we used a "proxy" to intercept and forward the requests from the proxy to the workers ---- On Tue, 22 Apr 2025 05:56:12 -0400 jamiedd13@aol.co.uk.INVALID wrote ---- Hi All,  Has anyone managed to get a REST extension working with multiple connect instances / workers? Many Thanks,  Jamie On Wednesday 16 April 2025 at 10:57:01 BST, Jamie <mailto: jamiedd13@aol.co.uk > wrote: Hi All,  I'm trying to implement a custom REST extension for Kafka Connect that allows a token to be used for authentication.  When I have a single instance of Kafka connect running in distributed mode ...

Hi All,  Has anyone managed to get a REST extension working with multiple connect instances / workers? Many Thanks,  Jamie On Wednesday 16 April 2025 at 10:57:01 BST, Jamie < jamiedd13@aol.co.uk > wrote: Hi All,  I'm trying to implement a custom REST extension for Kafka Connect that allows a token to be used for authentication.  When I have a single instance of Kafka connect running in distributed mode (i.e. 1 worker) this works as expected. However, when I add another instance (another worker) I get an error when the worker that received the request has to forward it on to the group leader: Error IO error forwarding REST request: (org.apache.kafka.connect.runtime.rest.RestClient:143) java.util.concurrent.ExecutionException: org.eclipse.jetty.client.HttpResponseException: HTTP protocol violation: Authentication challenge without WWW-Authenticate header  I noticed that someone also had the same issue with the basic authentication...

Yes, since Kafka 3.8, there is a so called "state updated thread". https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/DefaultStateUpdater.java#L183 -Matthias On 4/17/25 6:58 AM, Chad Preisler wrote: > In the latest version of Kafka Streams does KTable restoration happen in > its own thread? Can you point me to where that happens in the code base? > > Thanks, > Chad >

Hey folks, Just wanted to share something I thought might be useful to others in the community who are experimenting with or building on GenAI, LLMs, or RAG systems. 📅 *AI Learning Week* (April 28 – May 1) is a free 4-day webinar series organized by Xebia Academy. It covers both strategy and engineering sides of AI—but if you're more on the *builder* side like me, the *Tech Track* is where it gets really interesting. 🔧 Here's what's on the Tech Track: - *Building LLM-Powered Apps (Python)* – learn how to interface with LLMs, and spin up your first RAG + Agent - *Prompt Engineering Tips* – techniques to get clearer, more consistent results from tools like ChatGPT or Claude - *PDFs in RAG* – how to use multimodal models to parse complex PDF content (think tables, figures) and feed cleaner chunks into vector databases All sessions are *1 hour long*, with *zero marketing fluff*—just focused, practical insights. S...