Skip to main content

Relation between fetch.max.bytes, max.partition.fetch.bytes & max.poll.records

The doc states that fetch.max.bytes & max.partition.fetch.bytes

are not absolute maximum. If the first record batch in the first non-empty
> partition of the fetch is larger than this limit, the batch will still be
> returned to ensure that the consumer can make progress.


I am getting a bit confused.

Let's say I have a configuration like below with sufficient messages in
each partition


- Partitions in a topic 20
- Single message size 2MB
- Consumers 5
- max.poll.records 20
- fetch.max.bytes 50 MB
- max.partition.fetch.bytes 1 MB.

The broker config message.max.bytes and max.message.bytes is set to default
100MB

If the consumer does a poll will it receive 20 records? If yes then there
is no significance of fetch.max.bytes & max.partition.fetch.bytes with
max.poll.records?


- Java Kafka Client - 3.5.1
- Kafka Broker - 2.8.1

Comments