Kafka

Sunil, hi. Check out my blog maybe - I did have some problems getting the complete end to end trace data working in OpenSearch (it needed Data Prepper) - there may be similar issues with your stack. And as Kafka broker data/spans is missing, you have to infer time spent in Kafka from client data - not ideal I know! Regards, Paul From: sunil chaudhari <sunilmchaudhari05@gmail.com> Date: Wednesday, 11 November 2026 at 4:15 pm To: users@kafka.apache.org <users@kafka.apache.org> Cc: Brebner, Paul <Paul.Brebner@netapp.com> Subject: Re: Kafka tracing by Otel EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Thanks Paul for the quick response. We have a telemetry elastic cluster. There are kafka clients both producers and consumers already instrumented. My problem is.... On the service map on kibana , when I see the graph of clients connecting to Kakfa, if i click kafka icon, i see only the service latency, throughput and failed tr...

Thanks Paul for the quick response. We have a telemetry elastic cluster. There are kafka clients both producers and consumers already instrumented. My problem is.... On the service map on kibana , when I see the graph of clients connecting to Kakfa, if i click kafka icon, i see only the service latency, throughput and failed transaction rates. And those are all client metrics not the kafka one. I want to see kafka metrics in same servicemap coming via otlp protoocl. What I am going to try next: Kafka Broker- Jolokia agent- Edot efge collector( elastic distribution of open telemetry) - Straight to Telemetry Kibana. Does it make sense? will it show me kafka metrics to service map on Kibana? On Wed, 11 Nov 2026 at 10:03 AM, Brebner, Paul via users < users@kafka.apache.org> wrote: > Hi Sunil, > > I wish it was possible to introduce Otel tracing into Kafka brokers - this > would be extremely useful for a variety of purposes. > Unfortunatel...

Hi Sunil, I wish it was possible to introduce Otel tracing into Kafka brokers - this would be extremely useful for a variety of purposes. Unfortunately, it’s only possible for Kafka clients at present (e.g. a blog from last year https://www.instaclustr.com/blog/how-to-get-opentelemetry-traces-from-apache-kafka-clients-into-opensearch-using-data-prepper-and-opentelemetry-collectors/ ) The Kafka protocol isn’t supported by Otel at present. Regards, Paul From: sunil chaudhari <sunilmchaudhari05@gmail.com> Date: Wednesday, 11 November 2026 at 3:26 pm To: users@kafka.apache.org <users@kafka.apache.org> Subject: Kafka tracing by Otel EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hello kafka experts, Is it possible to instrument apache kafka broker using otel agent to monitor traces of incoming and outgoing clients to kafka?

For anyone based in Sydney, there’s still an opportunity to attend our first ever Australian Community over Code in Sydney next week! There are three Kafka talks: Paul Brebner, "AI for Kafka, and Kafka for AI”, 19 November, Room 1. PoAn Yang, "How Apache Kafka handles CPU/Memory in New Consumer Protocol”, 19 November, Room 1. Ming-Yen Chung, “From Now On, I Want Every Single Message”, 19 November, Room 1. The tickets are cheap, and if you are student/academic they are only $20! Register here: https://communityovercode.apache.org/events/sydney-2026/ I hope to see you there, Paul Brebner -- NetApp Instaclustr Technology Evangelist Read my blogs at https://www.instaclustr.com/paul-brebner/ Follow me on https://www.linkedin.com/in/paul-brebner-0a547b4/

I feel like that is not necessary, I tend to look at the current documentation for open source tools usually unless I am explicitly working with something that requires an older version, so I guess the ssl guide is enough for 4.4.0 On Tue, 27 Oct 2026 at 12:10, Gergely Harmadás <harmadasg@gmail.com> wrote: > Hi Chia-Ping, > > The plan sounds good to me. Since the document is really outdated, should > we also backport it to older supported releases like 4.3 and 4.2, what do > you think? > > > Thanks, > Gergely > > On Tue, 27 Oct 2026 at 03:48, Chia-Ping Tsai <chia7712@apache.org> wrote: > > > I haven’t looked at the PR yet: > https://github.com/apache/kafka/pull/23907 > > > > Doc fixes don’t need to wait for a release. Once the PR gets merged, we > > can file a patch to kafka-site to update the web site right away :) > > > > On 2026/10/21 22:01:28 Gergely Harmadás wrote: ...

Severity: important Affected versions: - Apache Kafka 0.11.0.0 through 4.1.2 - Apache Kafka 4.2.0 through 4.2.1 - Apache Kafka 4.3.0 through 4.3.1 Description: When a broker decodes a compressed record batch, the per-record decoder reads a declared body size from the decompressed stream and allocates a buffer of that size before validating it against the bytes available or any configured limit. The size is attacker-controlled, and the batch CRC does not constrain it, so a small (around 90-byte), CRC-valid compressed produce request can declare a body size of up to about 2 GiB and force the broker to attempt that allocation, raising OutOfMemoryError. The record size limit (max.message.bytes), the request size limit, and client/broker quotas bound only the compressed wire size, not the declared decompressed size. Impact: An authenticated principal with Write on a topic can crash broker request-handler threads (reached when a batch's codec differs from ...

Severity: important Affected versions: - Apache Kafka 0.10.0.0 through 4.1.2 - Apache Kafka 4.2.0 through 4.2.1 - Apache Kafka 4.3.0 through 4.3.1 Description: Several Kafka deserialization paths read a size or count field from input data and use it directly to allocate an array, collection, or byte buffer, with no check against the bytes actually available. A small crafted input can therefore declare a very large size and force a multi-gigabyte allocation. The affected paths are, in Kafka Streams, ProcessorRecordContext (header count), ChangedDeserializer (varint value length), CombinedKeySchema (foreign-key length, which could also produce a negative size), ProcessorMetadata (key length), and AssignmentInfo group-assignment decoding; and, in the clients library, ListDeserializer (used by ListSerde). Impact: An attacker can trigger the oversized allocation and raise an OutOfMemoryError that crashes the affected consumer or Kafka Streams thread; the b...