Kafka

Sunil, hi. Check out my blog maybe - I did have some problems getting the complete end to end trace data working in OpenSearch (it needed Data Prepper) - there may be similar issues with your stack. And as Kafka broker data/spans is missing, you have to infer time spent in Kafka from client data - not ideal I know! Regards, Paul From: sunil chaudhari <sunilmchaudhari05@gmail.com> Date: Wednesday, 11 November 2026 at 4:15 pm To: users@kafka.apache.org <users@kafka.apache.org> Cc: Brebner, Paul <Paul.Brebner@netapp.com> Subject: Re: Kafka tracing by Otel EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Thanks Paul for the quick response. We have a telemetry elastic cluster. There are kafka clients both producers and consumers already instrumented. My problem is.... On the service map on kibana , when I see the graph of clients connecting to Kakfa, if i click kafka icon, i see only the service latency, throughput and failed tr...

Thanks Paul for the quick response. We have a telemetry elastic cluster. There are kafka clients both producers and consumers already instrumented. My problem is.... On the service map on kibana , when I see the graph of clients connecting to Kakfa, if i click kafka icon, i see only the service latency, throughput and failed transaction rates. And those are all client metrics not the kafka one. I want to see kafka metrics in same servicemap coming via otlp protoocl. What I am going to try next: Kafka Broker- Jolokia agent- Edot efge collector( elastic distribution of open telemetry) - Straight to Telemetry Kibana. Does it make sense? will it show me kafka metrics to service map on Kibana? On Wed, 11 Nov 2026 at 10:03 AM, Brebner, Paul via users < users@kafka.apache.org> wrote: > Hi Sunil, > > I wish it was possible to introduce Otel tracing into Kafka brokers - this > would be extremely useful for a variety of purposes. > Unfortunatel...

Hi Sunil, I wish it was possible to introduce Otel tracing into Kafka brokers - this would be extremely useful for a variety of purposes. Unfortunately, it’s only possible for Kafka clients at present (e.g. a blog from last year https://www.instaclustr.com/blog/how-to-get-opentelemetry-traces-from-apache-kafka-clients-into-opensearch-using-data-prepper-and-opentelemetry-collectors/ ) The Kafka protocol isn’t supported by Otel at present. Regards, Paul From: sunil chaudhari <sunilmchaudhari05@gmail.com> Date: Wednesday, 11 November 2026 at 3:26 pm To: users@kafka.apache.org <users@kafka.apache.org> Subject: Kafka tracing by Otel EXTERNAL EMAIL - USE CAUTION when clicking links or attachments Hello kafka experts, Is it possible to instrument apache kafka broker using otel agent to monitor traces of incoming and outgoing clients to kafka?

For anyone based in Sydney, there’s still an opportunity to attend our first ever Australian Community over Code in Sydney next week! There are three Kafka talks: Paul Brebner, "AI for Kafka, and Kafka for AI”, 19 November, Room 1. PoAn Yang, "How Apache Kafka handles CPU/Memory in New Consumer Protocol”, 19 November, Room 1. Ming-Yen Chung, “From Now On, I Want Every Single Message”, 19 November, Room 1. The tickets are cheap, and if you are student/academic they are only $20! Register here: https://communityovercode.apache.org/events/sydney-2026/ I hope to see you there, Paul Brebner -- NetApp Instaclustr Technology Evangelist Read my blogs at https://www.instaclustr.com/paul-brebner/ Follow me on https://www.linkedin.com/in/paul-brebner-0a547b4/