ESXi 6 Storage vMotion on 3Par 7200c through iSCSI

[DAVIDsTEA]

Hi Guys,

We're in the process of setting up a new environment with ESXi 6 and a 3Par 7200c all-flash SAN through iSCSI.  So far everything is working fine and the results are pretty good with some nice IOPS and Bandwidth but I'm having some weird results with Storage vMotion that i'm wondering about.

Here's the situation.

If I have my vMotion setup on some 10GbE adapters and perform a Storage vMotion of a 120GB VM from a 3Par Volume to another, it takes about 7 minutes.  Even though it should take about 2 minutes at that speed, it takes 7.  I tried reconfiguring my vMotion to use 1GbE adapter but when I do the transfer it takes the same amount of time.  I also don't see any traffic going to the 3Par when I do the migration so I'm wondering if VMWARE and 3Par are intelligent enough to know that the 2 volumes are on the same SAN and coordinate to not use the network?  If so, that's great but why would it take 7 minutes for the move?

UPDATE:  I just read that 3Par supports VAAI so it seems that it's passing the commands to the 3Par rather than transferring on the network.  This is great news but I still don't understand why it would take 7 minutes to complete.  I know that this is probably a 3Par forum question now but since it's a popular SAN and involves VAAI, maybe someone can still guide me?

Any information would be helpful.

Thanks.

Nick.

Message was edited by: DAVIDsTEA

[Techie01]

Can you please check if VAAI is being used ( Refer http://www.yellow-bricks.com/2012/12/20/using-esxtop-to-check-vaai-primitive-stats/)

I am not sure how were able to  reach to the number of 2 minutes as expected time. Even though you have a 10gig network, are you sure that your end to end network is 10gig. Are you sure that your storage can transfer the speed to match the 2 minute transfer time.

Please look at the esxtop  for network, storage , CPU etc to check how the packet is flowing and if there is any other parameter getting invovled.

[RichardBush]

Could you confirm you have gone over the 3par best practices / setup guide for ESX ?

What OS version is the 7200 running ?

What host personas are you using ?

From the ESX hosts could you run the following : esxcli storage core device vaai status get or esxcfg-scsidevs -l

The only time i have seen it not work is, different block sizes on source and destination VMFS volume, a change to the VMDK layout (thick to thin). I am sure there will be more limitations listed in the HP guide too.

let me know the output and if all the settings have been applied from the documentation.

Rich

[dhanarajramesh]

I would suggest you to go thru http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-Storage-API-Array-Integration.pdf

and would suggest you to check whether VAAI is enabled and supported and if yes check with array vendor whether you can change the transfer rate size? refer page 12 and 13.

[DAVIDsTEA]

I'm running the latest OS on the 7200.

Not sure what you mean about host personas?

With the commands you put, all VAAI statuses show as supported.

At the moment, everything seems to be running well and performance seems great so I'm guessing VAAI is running well.

I was told by HP tech that the best bed for VM disks is to set them as Think Eager Zero since 3Par will handle the 0's instead of VMWARE.  Is this correct cause I'll be moving all our existing VM's to the new infra and I was planning to switch all the VMDK's to that format.

[DAVIDsTEA]

The end to end is 10gig but I just realized since I'm in a temporary setup, my network cables may not all be 10gig certified which may cause some issues.  I'll have to order those cables soon as see if it resolves anything.

I also found another bug with the Broadcom 10GbE NIC's and the "Delayed ACK" setting that was causing high read latency on the 3Par.  When I disabled this on the iSCSI adapters, the latency went from an average of 30-40ms to usually under 1ms.

[RichardBush]

Host persona (maybe referenced as host definition) is set on the 3par console per host, it should be set to 11 i believe for ESX 5 and up (check in the 3par implementation documents for ESX), in that document you also find a lot of advanced settings that need to be configured such as I/O queue depth etc.

http://h20628.www2.hp.com/km-ext/kmcsdirect/emr_na-c03290624-20.pdf

Do you have Jumbo frames set end to end ? (this shouldnt affect vMotion but may improve access speed)

Yes that's how i configure my Disks, as the 3par ASIC will see the zeros, so you wont loose space. Do you have while space in the VMFS volumes at the moment ? 3par guys can run peer motion from some arrays to do this copy for you, but i, like you, prefer to use Vmotion between the array's.

Anyhow the above document should walk you through all the setting you need to double check.

Cheers

Rich