0 Replies Latest reply on Aug 25, 2019 1:26 AM by ManivelR

    VSAN 6.7 U1 :- High IO latency from simple/large file copy

    ManivelR Enthusiast

      Hi All,

       

      VSAN :- High IO latency from simple/large file copy:

       

      VSAN setup is doing good in normal load.

       

      Here the issue started:-

       

      When we test using the following dd commands in Linux VM,I'm seeing very high latency.Its going to 500 ms,800 ms and sometimes it is going up to 1000 ms.Sometimes back,we tested the setup(before prod) with HCI benchmark and did not see this kind of high latency.

       

      This command is always slower as it's ignoring cache:- while :; do time dd if=/dev/zero of=/test.tmp bs=4M count=1000 oflag=direct ; time rm -v /test.tmp; done

       

      This command is usually really fast as it uses  cache:- while :; do time dd if=/dev/zero of=/test.tmp bs=4M count=1000; time rm -v /test.tmp; done

       

      This is another command  for simple reads is:- while :; do hdparm -Tt /dev/sda; done

       

      Tested with 2 VSAN setups.

       

      1st VSAN setup:-

      VSAN VMkernel NIC is 2*10 G(active/passive setup with DVS). 2 DG/ALL FLASH VSAN(passthrough mode HBA)

       

      2nd VSAN setup:-

      VSAN VMkernel NIC is 2*10 G(active/passive setup with DVS). 2 DG/ALL FLASH VSAN(passthrough mode)

       

      Could someone advise where is the bottleneck and how to address this issue?

       

      Thanks,

      Manivel R