Hi
One major reason that Kafka is fast is because it is using sendfile()
for zero copy, as what it described at
https://kafka.apache.org/documentation/#producerconfigs,
This combination of pagecache and sendfile means that on a Kafka
cluster where the consumers are mostly caught up you will see no read
activity on the disks whatsoever as they will be serving data entirely
from cache.
However, when I ran tracing on all my kafka brokers, I didn't get a
single sendfile system call, why his discrepancy?
sudo ./syscount -p 126806 -d 30
Tracing syscalls, printing top 10... Ctrl+C to quit.
[17:44:10]
SYSCALL COUNT
epoll_wait 108482
write 107165
epoll_ctl 95058
futex 86716
read 86388
pread 26910
fstat 9213
getrusage 120
close 27
open 21
One major reason that Kafka is fast is because it is using sendfile()
for zero copy, as what it described at
https://kafka.apache.org/documentation/#producerconfigs,
This combination of pagecache and sendfile means that on a Kafka
cluster where the consumers are mostly caught up you will see no read
activity on the disks whatsoever as they will be serving data entirely
from cache.
However, when I ran tracing on all my kafka brokers, I didn't get a
single sendfile system call, why his discrepancy?
sudo ./syscount -p 126806 -d 30
Tracing syscalls, printing top 10... Ctrl+C to quit.
[17:44:10]
SYSCALL COUNT
epoll_wait 108482
write 107165
epoll_ctl 95058
futex 86716
read 86388
pread 26910
fstat 9213
getrusage 120
close 27
open 21
Comments
Post a Comment