I believe no matter the partition count exceeds the broker count, we can
always have the same number of consumer instances as the partition count.
So what I want to know is when two partition exists on the same broker, two
consumer instances will be talking to same broker, is that bad ?
张祥 <xiangzhang1128@gmail.com> 于2020年2月27日周四 下午2:20写道:
> Thanks. What influence does it have for consumers and producers when
> partition number is more than broker number, which means at least one
> broker serves two partitions for one topic ? performance wise.
>
> Peter Bukowinski <pmbuko@gmail.com> 于2020年2月26日周三 下午11:02写道:
>
>> Disk usage is one reason to expand. Another reason is if you need more
>> ingest or output throughout for your topic data. If your producers aren't
>> able to send data to kafka fast enough or your consumers are lagging, you
>> might benefit from more brokers and more partitions.
>>
>> -- Peter
>>
>> > On Feb 26, 2020, at 12:56 AM, 张祥 <xiangzhang1128@gmail.com> wrote:
>> >
>> > In documentation, it is described how to expand cluster:
>> >
>> https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion
>> .
>> > But I am wondering what the criteria for expand is. I can only think of
>> > disk usage threshold. For example, suppose several disk usage exceed
>> 80%.
>> > Is this correct and is there more ?
>>
>
always have the same number of consumer instances as the partition count.
So what I want to know is when two partition exists on the same broker, two
consumer instances will be talking to same broker, is that bad ?
张祥 <xiangzhang1128@gmail.com> 于2020年2月27日周四 下午2:20写道:
> Thanks. What influence does it have for consumers and producers when
> partition number is more than broker number, which means at least one
> broker serves two partitions for one topic ? performance wise.
>
> Peter Bukowinski <pmbuko@gmail.com> 于2020年2月26日周三 下午11:02写道:
>
>> Disk usage is one reason to expand. Another reason is if you need more
>> ingest or output throughout for your topic data. If your producers aren't
>> able to send data to kafka fast enough or your consumers are lagging, you
>> might benefit from more brokers and more partitions.
>>
>> -- Peter
>>
>> > On Feb 26, 2020, at 12:56 AM, 张祥 <xiangzhang1128@gmail.com> wrote:
>> >
>> > In documentation, it is described how to expand cluster:
>> >
>> https://kafka.apache.org/20/documentation.html#basic_ops_cluster_expansion
>> .
>> > But I am wondering what the criteria for expand is. I can only think of
>> > disk usage threshold. For example, suppose several disk usage exceed
>> 80%.
>> > Is this correct and is there more ?
>>
>
Comments
Post a Comment