Hi All,
I'm in a sub-optimal situation whereby I have some Kafka Streams apps
deployed to production, but the default replication factor set on the
brokers was 1 when they were first deployed.
As such, any state store changelog topics, and re-partition topics
therefore have RF 1 also.
I'm familiar with the bin/kafka-reassign-partitions.sh tool and how to use
that.
I've since also found the streams replication.factor
(StreamsConfig.REPLICATION_FACTOR_CONFIG) setting that can be supplied in
the streams apps config.
My questions are:
1. Will simply changing the value of REPLICATION_FACTOR_CONFIG and
re-deploying have any effect on already-created internal topics?
2. Conversely, should I just change the RF of the internal topics using
the re-assign-partitions tool? Is that safe to do whilst the apps are still
running?
3. (not Streams-specific) If a broker that was the leader of
partition(s) for a topic with RF 1 died (i.e. no replicas), and was
non-recoverable - would the only way to restore service to those partitions
be to delete the topic and re-create? (with data loss, understandably). I
couldn't seem to achieve this using preferred/unclean leader election, nor
using the re-assign-partitions tool, since that requires all brokers to be
healthy.
Many thanks in advance for any answers,
Marcus
I'm in a sub-optimal situation whereby I have some Kafka Streams apps
deployed to production, but the default replication factor set on the
brokers was 1 when they were first deployed.
As such, any state store changelog topics, and re-partition topics
therefore have RF 1 also.
I'm familiar with the bin/kafka-reassign-partitions.sh tool and how to use
that.
I've since also found the streams replication.factor
(StreamsConfig.REPLICATION_FACTOR_CONFIG) setting that can be supplied in
the streams apps config.
My questions are:
1. Will simply changing the value of REPLICATION_FACTOR_CONFIG and
re-deploying have any effect on already-created internal topics?
2. Conversely, should I just change the RF of the internal topics using
the re-assign-partitions tool? Is that safe to do whilst the apps are still
running?
3. (not Streams-specific) If a broker that was the leader of
partition(s) for a topic with RF 1 died (i.e. no replicas), and was
non-recoverable - would the only way to restore service to those partitions
be to delete the topic and re-create? (with data loss, understandably). I
couldn't seem to achieve this using preferred/unclean leader election, nor
using the re-assign-partitions tool, since that requires all brokers to be
healthy.
Many thanks in advance for any answers,
Marcus
Comments
Post a Comment