Re: Kafka with RAID 5 on. busy cluster.

RAID 5 typically is slower because Kafka is very write heavy load and that creates a bottleneck because writes to any disk require parity writes on the other disks.

-hans

> On Mar 28, 2020, at 2:55 PM, Vishal Santoshi <vishal.santoshi@gmail.com> wrote:
>
> Ny one ? We doing a series of tests to be confident, but if there is some
> data folks, who have had RAID 5 on kafka, have to share, please do.
>
> Regards.
>
>> On Mon, Mar 23, 2020 at 11:29 PM Vishal Santoshi <vishal.santoshi@gmail.com>
>> wrote:
>>
>> << In RAID 5 one can loose more than only one disk RAID here will be data
>> corruption.
>>>> In RAID 5 if one looses more than only one disk RAID there will be data
>> corruption.
>>
>> On Mon, Mar 23, 2020 at 11:27 PM Vishal Santoshi <
>> vishal.santoshi@gmail.com> wrote:
>>
>>> One obvious issue is disk failure toleration . As in if RF =3 on.normal
>>> JBOD disk failure toleration is 2. In RAID 5 one can loose more than only
>>> one disk RAID here will be data corruption. effectively making the broker
>>> unusable, thus reducing our drive failure toleration to 2 drives ON 2
>>> different brokers with the added caveat that we loose the whole broker as
>>> well ?
>>>
>>>
>>> On Mon, Mar 23, 2020 at 10:42 PM Vishal Santoshi <
>>> vishal.santoshi@gmail.com> wrote:
>>>
>>>> We have a pretty busy kafka cluster with SSD and plain JBOD. We
>>>> planning or thinking of using RAID 5 ( hardware raid or 6 drive SSD
>>>> bokers ) instead of JBID for various reasons. Hss some one used RAID 5 ( we
>>>> know that there is a write overhead parity bit on blocks and recreating a
>>>> dead drive ) and can share there experience on it . Confluent advises
>>>> against it but there are obvious ease one gets with RAID ( RAID 10 is to
>>>> expensive space wise ) .... Any advise /comments etc will be highly
>>>> appreciated.
>>>>
>>>> Regards.
>>>>
>>>>