vSwitch config and Vmotion

[VTorque]

Hi Everyone..

Got a situation with Vmotion and thought I would post while I am investigating..

Scenario:

2 node cluster

All Guest VMs on single Host (host A)

Other host (host B) with no VMs.

We have made a change to the networking on the host B. This change was around a reccomendation from VMware to alleviate some network/vmotion/HA issues we have been experiencing.

The change:

Previoulsy all (6) NIC ports were teamed and connected to a single vSwitch, with all LAN and VMware traffic on it (including SC and VMkernal) - this config was QA'd by VMware at design stage. This was the only vSwitch. This configuration is still in place on the host A.

On host B we have now separated the 2 on-board NIC ports, into their own team, and left the other 4 as they were. There are now 2 vSwitches on this host, one vSwitch the same as it was before the change, except that 2 pNICs have and the VMkernal port group have been removed. This host also has the new vSwitch dedicated to VMkernal.

Our intention is now to make this change on Host A also. The problem is that vMotion is now not working (times out at 10%). I need to vMotion my VMs to host B before I can make the change to host A.

Any thoughts on if vMotion should work in the current situation?

[admin]

1st thing to check would be the default gateway on the new VMKernel port group for VMotion.

The 10% thing is leading me to look there.

[vmmeup]

On Host B all the NICS were once part of one vSwitch (vSwitch0) and now two of them have been removed and placed into a second vSwitch(vSwitch1)...am I right so far? I'm going to assume all 6 nics were trunked to a physical switch or maybe split between two. If so did you remove the two nics you split off from the Trunk? Where does your service console reside, on vSwitch0?

If one switch you ahould have:

vSwitch0 - 4 Nics (TrunkA)

-Service Console

-PortGroups for VM's

vSwitch1 - 2 nics (TrunkB)

-vmkernel adapter

-IP Address

-Gateway of Service Console

Is the vmkernel adapter on the same IP subnet as that of the other server (HostA)?

[adam_midalia · Accepted solution]

OK - I just whipped this up in the lab and I think your problem is that the vSwitches have different names.

If your original vSwitch was vSwitch0 and that had the VMkernel, Console and VM network - then the host with the new config with have a vSwitch0 with Console & VM network, and a vSwitch1 with VMkernel.

The basic rule of thumb is that VMotion only works if the vSwitch and Port Group names are identical i.e. you can't VMotion using a VMkernel switch from vSwitch0 to vSwitch1.

FYI - make sure you never team NICs of different brand/type. This will cause dropped packets and network errors.

[VTorque]

Hi guys..

Just got in to the office and it is awesome to see some replies! thanks.

BrianG - Checked this and it was configured all good.

Vmmeup - correct (re second vswitch etc), and yes I did split the NICs from the Trunk. Yes, SC was on vswitch0 (on both Hosts)

Marked as helpful cause adam.midalia was just a little more succinct below.

adam.midalia - thanks - hit the nail right on the head!

[vmmeup]

Just for reference this is not true....you can vmotion between servers that have vmkernel on differnet vSwitches. What is important is that the vmkernel has a gateway where it can reach the service console. I actually have a server in one of my vmware clusters that is exactly this and it works no problem. This wasn't done on purpose, but I just haven't fixed it. There are benefits to keeping all your servers configured exactly the same, but it isn't so vmotion will work. You're right it will fail if the Port Groups aren't named identically, but it doesn't check the actual vlan assigned to it. So it is posible to have identical Port Groups with different vlan ID's be carefull of that. It will however fail the vmotion before it even begins if there is not a matching portgroup on the destination server, it will not allow it to try and start.

[VTorque]

Thats interesting, as the scenario you describe is exactly where we were at.. with no vmotion happening. Corrected this by configuring the vSwitch changes on the second host, and everything started working.

Perhaps this is one of the scenarios where it will work on some implementations, and not others.. I am beggining to find a few of these.

[vmmeup]

It's also possible that other things changed as a result of the vSwitch changes.....I always like to try and find the true root of a problem. Always a good idea to document all teh changes so you can isolate to know for sure if it happend again. Glad to hear it's working though.

[VTorque]

For sure.

I will be getting to the bottom of this one (hopefully!) and will post here to let you know..

Cheers