Hi Richard,
The beans path is:
kafka.log:name=Size,partition=<partition>,topic=<topic>,type=Log
I don't have a jmx_exporter at hand to test it at the moment but I
don't see anything obviously wrong in your config, other than type:
GAUGE missing.
Did you try browsing the beans with jmxterm before configuring the exporter?
Gabriele
On Mon, 24 Feb 2020 at 23:01, Richard Rossel <henhiskan@gmail.com> wrote:
>
> Hi Gabriele,
> I'm using Kafka 5.3.1, which is apache kafka 2.3 and I'm using JMX to
> retrieve metrics from brokers.
> The only metric I saw from "kafka.log" is LogFlushStats, but nothing
> about Log.partition
> This is the pattern I have deployed for kafka.log metrics, maybe I
> need a different for partitions sizes?
>
> - pattern : kafka.log<type=Log, name=(.+), topic=(.+),
> partition=(.+)><>Value
> name: kafka_log_$1
> labels:
> topic: "$2"
> partition: "$3"
>
> what do you think?
>
> On Mon, Feb 24, 2020 at 4:05 PM Gabriele Paggi <gabriele.paggi@gmail.com> wrote:
> >
> > Hi Richard,
> >
> > If you are running Kafka > 1.0.0 the information you are looking for
> > is exposed by the Admin API, describeLogDirs method, or via CLI with
> > kafka-log-dirs.sh.
> > The metrics are also exposed by the dropwizard metrics reporter, e.g.
> > for Graphite:
> > kafkalog001.kafka.log.Log.partition.49.topic.__consumer_offsets.Size.value
> > 1315862 1582578155
> >
> > Gabriele
> >
> > On Mon, 24 Feb 2020 at 19:54, Richard Rossel <henhiskan@gmail.com> wrote:
> > >
> > > Hi List,
> > > I'm trying to find the way to keep track of topics' size (or
> > > partitions) across brokers, but
> > > I didn't found any Kafka metrics for that. I start wonder if that is possible.
> > >
> > > I have ideas for a work around (querying log files) but I wonder what
> > > will be the right way or how do you keep monitoring this type of
> > > metric?
> > >
> > > Thanks.-
> > >
> > > --
> > > Richard Rossel
> > > Atlanta - GA
> >
> >
> >
> > --
> > Gabriele
>
>
>
> --
> Richard Rossel
> Atlanta - GA
--
Gabriele
The beans path is:
kafka.log:name=Size,partition=<partition>,topic=<topic>,type=Log
I don't have a jmx_exporter at hand to test it at the moment but I
don't see anything obviously wrong in your config, other than type:
GAUGE missing.
Did you try browsing the beans with jmxterm before configuring the exporter?
Gabriele
On Mon, 24 Feb 2020 at 23:01, Richard Rossel <henhiskan@gmail.com> wrote:
>
> Hi Gabriele,
> I'm using Kafka 5.3.1, which is apache kafka 2.3 and I'm using JMX to
> retrieve metrics from brokers.
> The only metric I saw from "kafka.log" is LogFlushStats, but nothing
> about Log.partition
> This is the pattern I have deployed for kafka.log metrics, maybe I
> need a different for partitions sizes?
>
> - pattern : kafka.log<type=Log, name=(.+), topic=(.+),
> partition=(.+)><>Value
> name: kafka_log_$1
> labels:
> topic: "$2"
> partition: "$3"
>
> what do you think?
>
> On Mon, Feb 24, 2020 at 4:05 PM Gabriele Paggi <gabriele.paggi@gmail.com> wrote:
> >
> > Hi Richard,
> >
> > If you are running Kafka > 1.0.0 the information you are looking for
> > is exposed by the Admin API, describeLogDirs method, or via CLI with
> > kafka-log-dirs.sh.
> > The metrics are also exposed by the dropwizard metrics reporter, e.g.
> > for Graphite:
> > kafkalog001.kafka.log.Log.partition.49.topic.__consumer_offsets.Size.value
> > 1315862 1582578155
> >
> > Gabriele
> >
> > On Mon, 24 Feb 2020 at 19:54, Richard Rossel <henhiskan@gmail.com> wrote:
> > >
> > > Hi List,
> > > I'm trying to find the way to keep track of topics' size (or
> > > partitions) across brokers, but
> > > I didn't found any Kafka metrics for that. I start wonder if that is possible.
> > >
> > > I have ideas for a work around (querying log files) but I wonder what
> > > will be the right way or how do you keep monitoring this type of
> > > metric?
> > >
> > > Thanks.-
> > >
> > > --
> > > Richard Rossel
> > > Atlanta - GA
> >
> >
> >
> > --
> > Gabriele
>
>
>
> --
> Richard Rossel
> Atlanta - GA
--
Gabriele
Comments
Post a Comment