Trouble getting vMotion to work on dedicated/isolated switch

[Xourque]

Hello!

In advance, thank you to anyone who can help me with this.

Request To All of You:

Please let me know what I haven't set up right. I'm obviously oblivious to what I didn't do right!

Issue Statement:

If vMotion is enabled on interfaces connected to a dedicated/isolated switch, vMotion fails to move a guest OS between hosts.

Error Message:

The vMotion migrations failed because the ESX hosts were not able to connect over the vMotion network. Check the vMotion network settings and physical network configuration.

vMotion migration [-1062731773:1352746564898648] failed to create a connection with remote host <192.168.0.1>: The ESX hosts failed to connect over the VMotion network

Migration [-1062731773:1352746564898648] failed to connect to remote host <192.168.0.1> from host <192.168.0.3>: Timeout

The vMotion failed because the destination host did not receive data from the source host on the vMotion network. Please check your vMotion network settings and physical network configuration and ensure they are correct.

Details:

- It works perfectly fine if I connect the vMotion enabled interfaces to the shared corporate switch (the same one that shares guest OS traffic).

- vSphere 5.1.0, 838462

- The switch is a Dell PowerConnect 2816 in managed mode.

- The switch is physically isolated from the rest of my network. The only devices plugged into it are four cables (two each going to an ethernet port on two hosts), and to a test computer for pinging purposes.

- I followed VMWare KB article 2007467.

- I currently have a patch cable going between the vMotion switch and the corporate switch, so that I can log into the switch and do diagnostics. So if you see the "Networks" column below showing the 172.20.11.0 network, that's the corporate network, where as I've assigned each vMotion VMKernel a 192.168.0.0 IP.

- I have a computer connected to the switch (192.168.0.200), and from there, I can ping 192.168.0.1, .2, .3, and .4, which are all IPs assigned to each VMKernel port (see screen shots below).

- From the Dell PowerConnect switch, I can ping 192.168.0.200 (my test computer), but I can't ping any of the VMKernal port IPs....Strange!

What I've Tried:

- Deleting, then re-adding the vSwitch on each ESXi host, thinking I did something wrong.

- Resetting the Dell PowerConnect 2816 to defaults, then setting it back up. All I've done to the switch is give it an IP of 192.168.0.2. I've set nothing else, such as VLANs, jumbopackets, or anything else of that nature. It's literally at defaults with only the management IP added.

- Connecting each VMKernal assigned ethernet port (for vMotion) to the corporate switch, and vMotion worked just fine.

Screen Shots Showing Configuration:

ESXi 5.1 host #1	ESXi 5.1 host #2
Showing the vSwitch, its two VMKernal ports, and vmnic3&5:	Showing the vSwitch, its two VMKernal ports, and vmnic4&5:
Showing the general tab of VMKernal "vMotion01":	Showing the general tab of VMKernal "vMotion01":
Showing the NIC Teaming tab of VMKernel "vMotion01":	Showing the NIC Teaming tab of VMKernal "vMotion01":
Showing the General tab of VMKernel "vMotion02":	Showing the General tab of VMkernel "vMotion02":
Showing the NIC Teaming tab of VMKErnel "vMotion02":	Showing the NIC Teaming tab of VMKernel "vMotion02":

[spravtek]

Although you provided a lot of info, I'm still not seeing exactly how your vMotion pNics are connected to the pSwitch ...

Another thing, Is there a reason why on host 1 you have "(All) 4095" enabled under VLAN ID?

Thirdly, is this a typo, or did you give the pSwitch 192.168.0.2 as IP-address, same as one of the vMotion vmkernel??

- Resetting the Dell PowerConnect 2816 to defaults, then setting it back up. All I've done to the switch is give it an IP of 192.168.0.2. I've set nothing else, such as VLANs, jumbopackets, or anything else of that nature. It's literally at defaults with only the management IP added.

[Xourque]

Thanks for your reply!

1) What exact detail can I provide for that?

2) Those VLAN settings were me experimenting. Right now, both hosts have the NONE option chosen.

3) Typo. The switch is 192.168.0.10.

[jdptechnc]

It's not something that I regularly use, but as far as I know the main use case for using the 4095 VLAN on a vSwitch is when you're using VLAN tagging and passing all traffic from all VLANs to a VM for the tagging.  I think for your vmk ports, this is a misconfiguration.  Set it back to 0 like your other host.

[jdptechnc]

I see from the NIC Teaming tab that the interfaces on the hosts can only see a 172.20.x.x network.  What are these?  Is it possible that your switch ports are configured for this 172 network and blocking your 192.168 traffic?

[spravtek · Accepted solution]

- Are you able to reach one of the Vmotion addresses from the host itself, through CLI (ssh into the host)?

For example, your host is 192.168.0.20 and you ping the vMotion that is configured on that same host which is IP 192.168.0.2 ...

- What is the vmkernel gateway?

- you have 2 cables connected to the pNics which are connected to the vMotion vSwitch, and these 2 cables are on the same pSwitch where the 2 cables of the other host are connected (which are also connected to the pNics connected with the vMotion vSwitch on that host)....

[Xourque]

Ah. Well. Now I feel like an idiot. The "VMKernel Default Gateway:" was set to 172.20.1.1.

I set it to 192.168.0.100 (random number I chose) and now vMotion works just fine.

I did notice that it changed ALL of my VMKernels to that gateway, on every vSwitch, including the management and guest vSwitches (which are on the 172.20.1.0 network).

I need to read up on that VMKernel Default Gateway variable to see what this all means...

THANKS!!!!!