Hi Team,
I am currently operating a Kafka cluster in KRaft mode and would like to
raise a few questions and suggestions regarding the cluster scale-down
process.
Recently, I scaled down the cluster by removing one broker; however, I
encountered a situation where the removed broker still appeared in the
cluster, causing new partitions to be assigned to it. To address this, I
was able to remove the broker from the controller metadata using the
kafka-cluster.sh command. However, I could not find any documentation on
this method, so I had to refer to the Kafka source code to discover this
solution. The issue is now resolved, but I would like to ask if there is a
more streamlined approach for scaling down.
Additional Questions and Suggestions:
1.
*Automatic Cluster Exclusion*
I initially expected that a removed broker would be automatically
excluded from the cluster after a certain period. However, the broker
remained part of the cluster for quite some time. I am curious whether
Kafka has a feature that automatically detects and excludes removed brokers
from the cluster.
2.
*Atomic Scale Down*
If the above feature is not available, I would like to inquire if there
is a way to atomically exclude a broker and simultaneously remove it from
the cluster metadata. Since we manage a Kafka cluster with continuous topic
creation, a non-atomic process might lead to new topics being assigned to
the broker between its exclusion and metadata removal, necessitating manual
partition reassignments.
3.
*Documentation Enhancement (Suggestion)*
While addressing the scale-down issue, I could not find any relevant
information in the Kafka User Guide. I had to refer to the source code,
specifically the unregister functionality, to learn that this could be
done using kafka-cluster.sh. Including documentation on this process
would likely make it easier for other users to manage their clusters
effectively.
Thank you, and I look forward to any guidance or recommendations.
Best regards,
hayoung
I am currently operating a Kafka cluster in KRaft mode and would like to
raise a few questions and suggestions regarding the cluster scale-down
process.
Recently, I scaled down the cluster by removing one broker; however, I
encountered a situation where the removed broker still appeared in the
cluster, causing new partitions to be assigned to it. To address this, I
was able to remove the broker from the controller metadata using the
kafka-cluster.sh command. However, I could not find any documentation on
this method, so I had to refer to the Kafka source code to discover this
solution. The issue is now resolved, but I would like to ask if there is a
more streamlined approach for scaling down.
Additional Questions and Suggestions:
1.
*Automatic Cluster Exclusion*
I initially expected that a removed broker would be automatically
excluded from the cluster after a certain period. However, the broker
remained part of the cluster for quite some time. I am curious whether
Kafka has a feature that automatically detects and excludes removed brokers
from the cluster.
2.
*Atomic Scale Down*
If the above feature is not available, I would like to inquire if there
is a way to atomically exclude a broker and simultaneously remove it from
the cluster metadata. Since we manage a Kafka cluster with continuous topic
creation, a non-atomic process might lead to new topics being assigned to
the broker between its exclusion and metadata removal, necessitating manual
partition reassignments.
3.
*Documentation Enhancement (Suggestion)*
While addressing the scale-down issue, I could not find any relevant
information in the Kafka User Guide. I had to refer to the source code,
specifically the unregister functionality, to learn that this could be
done using kafka-cluster.sh. Including documentation on this process
would likely make it easier for other users to manage their clusters
effectively.
Thank you, and I look forward to any guidance or recommendations.
Best regards,
hayoung
Comments
Post a Comment