2 Replies Latest reply on Oct 17, 2020 9:25 AM by alphenit

    Esxi 6.7 ssd i/o latency spikes

    Pollard Novice

      Seeing latency spikes in excess of 100ms on my 2 samsung 860 evo ssd's. The throughput is fine getting 500Mbps+ on them but the vmkwarning.log has "performance has deteriorrated. I/O latency increasted from average value" warnings. This is a home server thats why I'm using regular samsung ssd's.

       

      ESXi version: 6.7.0

      ESXi building number: 15160138

        • 2. Re: Esxi 6.7 ssd i/o latency spikes
          alphenit Enthusiast

          I've seen similar issues (high latency spikes of 200ms ) with even the latest 6.7U3 as of September 2020. This in combination with Samsung EVO 860 SSD.
          Which driver is your sata-port using?
          Have you tried disabling the vmware ahci driver? (below command and a reboot will make ESXi use the ahci driver instead of the vmw_ahci driver).
          you can check this under storage adapters through the host ui. When using the ahci driver you should be seeing ALL your sata ports. (the vmw_ahci only showed me 1 or 2).

           

          esxcli system module set --enabled=false --module="vmw_ahci"
          https://anthonyspiteri.net/homelab-supermicro-5020d-tnt4-storage-driver-performance-issues-and-fix/


          Also did some some dd tests after cd-ing into the specific vmfs volume but not too sure what that signifies or how to interpret the results(saw it used in another thread) and using ahci driver made the dd command finish quite a lot faster:

          using vmw_ahci:

          time sh -c "dd if=/dev/zero of=test bs=16k count=640k conv=sync"

          655360+0 records in

          655360+0 records out

          real    2m 42.31s

          user    0m 0.00s

          sys     0m 0.00s

           

          using ahci:

          time sh -c "dd if=/dev/zero of=test bs=16k count=640k conv=sync"

          655360+0 records in

          655360+0 records out

          real    0m 47.97s

          user    0m 0.00s

          sys     0m 0.00s

           

          Wish VMware would fix those issues with vmw_ahci permanently so admins don't have to troubleshoot their own Lab hardware during the weekends.

           

          Consider marking this response as "Correct" or "Helpful" if you think my response helped you in any way.