Hi,
instead of tweaking retention time/size, you can try using
log.cleaner.enable=false. It's true by default as of 0.9.0.1. However, i
haven't tried that myself.
There is always going to be an impact since it takes up resources. And in
production, the general idea is to do housekeeping at an acceptable rate.
That again, depends on how much cost you can bear.
Thanks,
On Tue, 30 Jul 2019 at 11:29, Nayak, Soumya R. <snayak@firstam.com> wrote:
> Hi Team,
>
> There are two variables in Kafka - log.retention.hours and
> log.retention.bytes.
> What is the value has to be set for the above two variables so that there
> is no limit of hours or bytes, the messages stays forever? Is it a
> production level set up ?
> Will it impact kafka performance and also use much of disk space or can we
> in between change the above variable values and restart the brokers?
>
> Why I am asking the above is in hyperledger fabric blockchain framework
> kafka and zookeeper is used as a messaging service but recently many people
> have complained about its pruning of data for unknown reasons.
> So just wanted to know how the best can the kafka be configured so that
> there is no issues of pruning. Currently zookeeper version - 3.4.10 and
> kafka version - 1.0.0 is used. Will the issues be resolved migrating to
> latest versions?
>
> Regards,
>
>
>
> ******************************************************************************************
> This message may contain confidential or proprietary information intended
> only for the use of the
> addressee(s) named above or may contain information that is legally
> privileged. If you are
> not the intended addressee, or the person responsible for delivering it to
> the intended addressee,
> you are hereby notified that reading, disseminating, distributing or
> copying this message is strictly
> prohibited. If you have received this message by mistake, please
> immediately notify us by
> replying to the message and delete the original message and any copies
> immediately thereafter.
>
> If you received this email as a commercial message and would like to opt
> out of future commercial
> messages, please let us know and we will remove you from our distribution
> list.
>
> Thank you.~
>
> ******************************************************************************************
> FAFLD
>
instead of tweaking retention time/size, you can try using
log.cleaner.enable=false. It's true by default as of 0.9.0.1. However, i
haven't tried that myself.
There is always going to be an impact since it takes up resources. And in
production, the general idea is to do housekeeping at an acceptable rate.
That again, depends on how much cost you can bear.
Thanks,
On Tue, 30 Jul 2019 at 11:29, Nayak, Soumya R. <snayak@firstam.com> wrote:
> Hi Team,
>
> There are two variables in Kafka - log.retention.hours and
> log.retention.bytes.
> What is the value has to be set for the above two variables so that there
> is no limit of hours or bytes, the messages stays forever? Is it a
> production level set up ?
> Will it impact kafka performance and also use much of disk space or can we
> in between change the above variable values and restart the brokers?
>
> Why I am asking the above is in hyperledger fabric blockchain framework
> kafka and zookeeper is used as a messaging service but recently many people
> have complained about its pruning of data for unknown reasons.
> So just wanted to know how the best can the kafka be configured so that
> there is no issues of pruning. Currently zookeeper version - 3.4.10 and
> kafka version - 1.0.0 is used. Will the issues be resolved migrating to
> latest versions?
>
> Regards,
>
>
>
> ******************************************************************************************
> This message may contain confidential or proprietary information intended
> only for the use of the
> addressee(s) named above or may contain information that is legally
> privileged. If you are
> not the intended addressee, or the person responsible for delivering it to
> the intended addressee,
> you are hereby notified that reading, disseminating, distributing or
> copying this message is strictly
> prohibited. If you have received this message by mistake, please
> immediately notify us by
> replying to the message and delete the original message and any copies
> immediately thereafter.
>
> If you received this email as a commercial message and would like to opt
> out of future commercial
> messages, please let us know and we will remove you from our distribution
> list.
>
> Thank you.~
>
> ******************************************************************************************
> FAFLD
>
Comments
Post a Comment