VM's Losing Network When Migrated to New ESXi Host

[davisjj]

Ok, this has been a tough one to google (or the week has just been too long), but...

We have a 10 host cluster, ESXi 5.1, with SAN storage.  We also use a vSphere Distributed Switch with eleven port groups (for each vlan, management, and vMotion).  Each server has eight 1Gb physical network ports.

This week we replaced two of our host servers (i.e. new hardware).  I went through the normal process of building the ESXi box, network/storage config, etc.  Initially, I connected one physical NIC to a vSphere Standard Switch and then after the machine was built, added it to the dvSwitch, migrated the management connection to a virtual adapter, created the vMotion virtual adapter, etc. (the same thing that I have done several times before).

Because the new servers were added to the same dvSwitch as the existing servers, they automatically see the existing port groups.  The problem is that whenever I try to migrate a VM to one of the new servers, the VM loses its network link as soon as vMotion completes.  The only way for me to reconnect the VM is to either migrate it to one of the eight existing hosts or I can right click on the dvSwitch, Manage Hosts, choose the problem host, and skip through to "Migrate virtual machine networking," expand the VM to expose the network adapter, and then migrate the adapter to the proper destination port group (the SAME one that it was already assigned to).

This happens repeatedly.  I can then move the VM off and back onto that same host and it will continue to function.  If I go to migrate another VM (with the same port group, another port group, etc), I will have to go through the same process.  This has not happened in the past when replacing ESXi hosts.  Has anyone run across this?  Is there something that I am missing?

Thanks,

Jesse

[ep4p]

Have you tried removing the hosts from the vDS switch and re-add them?

[davisjj]

Yes, I have removed and added the hosts, multiple times actually.  I have blown away the network configuration on the host and recreated the virtual adapters (again multiple times).  It's getting very frustrating...

[ep4p]

After migration is VM's network status still connected? As well what do you see under the switch port state?

[a_p_]

Did you already check the physical switch port's configuration to ensure it's not a spanning-tree issue?

André

[davisjj]

Yes.  The switch configuration actually never changed.  We swapped out one server for another, same number of ports, etc.  

[davisjj]

If I look at it from the networking screen, I have to refresh, but it shows the link state as down.

[davisjj]

Also, if I go to the affected VM's settings and change the network adapter to another port group, apply that, and then change it back, it starts working.  I tried simply disconnecting the adapter and reconnecting and that did not work.

[davisjj]

More information...  All hosts are connected to a four-switch stack of Cisco Catalyst 3750-G's.  All ports are trunked (8 for each server).  No etherchannel is being used.

I did a "show mac-address | incl [virtual mac of VM]" and saw that when I moved a VM to one of the new servers, the switch did NOT get notified that the VM should be associated with one of the physical ports attached to the new host (i.e. it was still showing as being seen on a physical port connected to the host that I just migrated FROM).  If I migrated the VM to any of the existing hosts, the switch updated immediately and showed the MAC address to be on a different port.

I am going to do some digging into the settings that have to do with notifying the switch, MAC/ARP tables, etc.  I thought that was all managed at the dvSwitch or Port Group level, but maybe there is something at the host level that is causing me an issue?  Worse yet, I hope it is not related to the particular NICs in use on these servers.  They are different than what the existing servers were using.  I could do some testing on that (the hard way) I suppose.  I am running out of ideas, but at least I think I have a bit more information than I did.

Any other thoughts?

[davisjj]

Ok, now I feel silly...  I think I may have resolved the issue.  When I built the server, I followed the instructions and downloaded the ESXi image specific to HP.  It turns out that this was only ESXi 5.0 and the existing servers are ESXi 5.1.  I updated one of the new servers to 5.1 and everything seems to be fine now.

[davisjj]

I updated the second server and it is working fine now as well.

[ehk007]

Hi,

I crated 2 VMs at home, and i sent To move thé host in my office, différent networks.

What I have To do,do I need To change Up adresses ?

Thanks in advance 

Regards

Sam

Emsil: ehk007@gmail.com

[scott28tt]

@ehk007 

Which VMware product are you using?

Why not create your own thread, since it seems you're asking something different to what this thread is about?