High vm's write latency

[marcinkloc]

Hello,

We have experienced problem with high vm's write latency on vmfs datastore (4x4TB RAID 5).The GAVG parameter on vmhba and disk has 2-10ms, but on vm view our vms has from 20 up to 60ms write latency. On another host  (6x4TB RAID50) with higher load we have write latency from 0 to 5ms for all vm's running on it. What can be the problem if GAVG is at acceptable level but vm write latency is high?

[RAJ_RAJ]

Hi ,

Could you please share what application is hosted on the VM .

Also how many VMs connected to this Data store and what is size of disk and type provisioning .

If you can share esxtop results it will be easy to check

You may refer the below for monitor the performance .

https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-65-monitoring-performance-...

[RParker]

We have experienced problem with high vm's write latency on vmfs datastore (4x4TB RAID 5).The GAVG parameter on vmhba and disk has 2-10ms, but on vm view our vms has from 20 up to 60ms write latency.

60 ms is nothing, that is basically not even 1/10th of a second, 100ms is 1/10 milli(1 /1000) of a second so 60 is fairly low.. maybe by comparison it seems higher but it depends on disk type, cache and load inside the VM..

latency isn't the problem its IOPS, you need to find out what IOPS are going on inside that guest machine response time is worrisome if you are over 100ms consistently then you might have a problem.. but under load a VM might have higher latency but it may mean the VM is merely doing something that it needs to read / write a lot.. what kinds of disks are these?

[marcinkloc]

Hi, thank you for your responses

In most, vm's are tomcat servers with java app on it.

For now, host has 8 vms running on datastore RAID5, vm's disks size from 40GB-300GB, datastore size 10,9TB / 9TB free, thick provision (in most)

I have esxtop screen before vms migration to another datastore (three hosts, from top: host1(RAID5), host2(RAID0), host3 (RAID50))

You can see that, when one vm generates some IO operations it affects all vms runing on datastore, and for comparision host3 generates almost same IOPS with larger data size and the write latency is low.

I know that RAID50 has better write performace than RAID5, and the raid controller cache can improve performace, but I don't know if the write latency at this IOPS level and RAID configuration is normal?