How many vmotion nics

[rz1]

Hello all,

We are in the process of moving from vSphere 4.1 to vSphere 5.1.  As part of this effort, we have  the opportunity to reconfigure a few things, including the number of nics we'll dedicate to vmotion.  My understanding is that it can support up to 16x 1gb nics. 

Our server environment is not at 10gb yet, but we have a total of 12x 1gb nic ports on most of our ESX hosts (4x on-board, and 2x 4-port cards) with ports to spare.

As we build out the new 5.1 environment, is there a best practice around how many ports to configure for vmotion?  I expect the answer will be "it depends", but I just wanted to get an idea of the factors that should be taken into consideration for this?

Would the number of vm's on each ESXi host be a consideration?  (For example, when you place it in maintenance mode.)

Do we just need to experiment with it ourselves to see what the right number (for us) is?

Is there a minimum number we should consider (perhaps 2 or 3)?  Should we just configure as many as we can?

Thanks for your help....

[elgreco81 · Accepted solution]

Hi,

You are right...it depends

VMotion operations are very fast with vSphere if you already have a 1Gb NIC dedicated to it on each host. So, if you have any planned downtime for any of your hosts, I think it wouldn't be any need for a second one. 8 vmotion operations at the same time could be great (if you had 10Gb)...but maybe is not worth it if you can wait like 30 seconds more...

If you have DRS, which uses VMotion to load balance your VMs across your hosts, maybe a second NIC would be a good idea...but again...if that configuration is not working properly with 1 NIC you should be seeing warnings by now in your vCenter.

So, what DO best practices talk about is how to provide the best possible solution to a scenario with certain limitations that you have to consider. Availability would be one of the things to consider and one down side for availability would be "cost of aquisition" (budget is other milestone to consider in every project). As you already have more NICs, that's not a limitation and a good design would consider having a second NIC dedicated to VMotion (two vswitches or one with the two NIC linked to it...depends if your host has CPU contention or not...if not, I would recomend two vSwitches so you have HA in there too).

VMotion (at least for me) is not critical...I would first confirm that you won't need that "extra" NIC for more critical traffic like VM or storage presentation.

This is only my point of view (with some VMware best practices on it ), hope it helped you!

Regards,

elgreco81

[elgreco81]

Hi,

Was my help useful? Please remember to award the available points or give us some feedback if you have further questions.

Thanks,

elgreco81

[rz1]

Yes it was elgreco81.  And thanks for your quick response!

[elgreco81]

Great! Thank you!

[HeathReynolds]

I use the amount of RAM per host, and the RAM per guest, to help determine how man vMotion interfaces.

We use 10g and we are normally doing 1 vmotion interface for systems with less than 512gb of ram, 2 for more. Our boxes with 1TB of RAM get 4 interfaces, but they aren't dedicated. Storage/vMotion/VMs ride all of the links and we use QOS at the fabric.

vMotion performance is much better going from 4.1 to a 5.x release thanks to Stun During Page Send. If you don't run guests larger than 32gb of RAM and don't use large hosts then one vmotion interface is fine.