Hello Gentlemen.
As part of our upgrade to ESX/vSphere 4.1, we also opted to reformat our datastores in order to use a VMFS blocksize of 8MB instead of the default 1MB. So we are emptying LUNs one using our dear storage VMotion functionality, reformat it and sVMotion it the VMs back.
During this procedure we noticed that the sVMotion between different VMFS blocksizes is slow as hell and causes huge DAVG/GAVG latencies in our environment. Some example numbers from tests we conducted:
All tests conducted with VMFS version 3.46 and an idle VM with a thick-VMDK of 8GB (about 4GB actually used inside the GuestOS). LUN1 has a blocksize of 1MB, while LUN2 and LUN3 have a blocksize of 8 MB.
LUN1 to LUN2 => Task took 245 seconds, average latency ~140ms
LUN2 to LUN3 => Task took 60 seconds, average latency ~13ms
LUN3 to LUN1 => Task took 526 seconds, average latency ~130ms
all LUNs reside on the same FC storage backend in (EVA8000) in the same disk group, but we noticed the same behavior on our EVA4000 too.
as per HP recommendation, we use the RoundRobin PSP and IOPS=1 ()
happens during cold migration when the VM is powered off too
nothing except this sVMotion is happening on the host, nothing particularly heavy is happening on the other hosts using the same LUNs
VMware support recommend to look into the Disk.DiskMaxIOSize Advanced setting () but I'm not sure if this even applies because in resxtop, we always see way smaller IOs during sVMotion (if I compare MB/s and CMDS/s or READS/WRITES/s).
In any case, I set Disk.DiskMaxIOSize it from the default value of 32768KB to 1024KB and tested again, but it didn't improve the sVMotion performance.
Anyone of you fine gentlemen got any idea?
--
http://alpacapowered.wordpress.com
