Hello everyone. I kind of ran into weird issue here. I work in a school district. We have a 3 node vSphere cluster. The nodes are Dell R740 servers. Each server has two SFP+ built-in Intel X710 ports. We also have 2 Intel X710 SFP+ cards installed in each host. We recently setup vMotion as well as properly setup the iSCSI traffic to go out over the 10GB connections. The last thing I was trying to do was switch the VM Client traffic from going out the management port and have it go out two of the 10GB SFP+ ports.
On each host I created a standard vSwitch that has two active 10GB physical nics attached to it. The teaming mode is set to “Route based on originating virtual port”. The vlans I am trying to trunk through these connections is vlan 6 and 10. Now in the server room at the HS we have an MDF access switch stack and our core switch stack. Switches are Alcatel. The two 10 GB connections for host 1 are connected to switch 1 and 2 of our mdf access switch. The other two hosts have each of their 10 GB connections going into switch 1 and 2 of our core switch. The MDF access switched is trunked into the Core Switch. I made sure that each port is configured for 802.1q tagging of the vlans above.
Now here is where I am getting stumped. Host 1 and 3 work fine. I was able to configure all VMs to the new port group (for vlan 6) with no issue and I can ping all VMs after doing that. I also tested removing each connection one at a time and everything stays up. Host 2 is where I am having an issue. The first VM I moved over was one of our DCs. It immediately would not ping at all and our SolarWinds Orion system started sending out the usual alerts that it was down. If I unplug the 10 GB connection going into core switch 1, I am able to ping it no problem. The second I reconnect that connection I am unable to reach it again. I then tried switching the two 10GB connections. So I took the 10 GB wire that went to switch 2 and put it in switch 1 and vice versa. After this the DC started pinging so I thought everything was resolved. Moved the next VM over (which was the vCenter appliance) and everything was good. Finally went to move the final VM which was one of our DFS servers, and now I can’t reach that server but the other two VMs are ok. Pulled out the wire going to Core switch 1 and low and behold the DFS server starts pinging. I try switching the wires again (so original switch 1 wire back to switch 1 and the same for 2). Once I did that the DFS server was pinging fine but now the DC and the vCenter appliance wouldn’t ping.
I have no idea what could be causing this. I right away was thinking network switch config issue but I have triple checked the ports and I can’t find anything different between them. For now I moved the vcenter applicance and the DFS server back to the management interface and left the DC on the port group with the switch 1 wire disconnected. I noticed that I had loopback detection enabled on the two 10 GB connections for this host and I thought maybe that was the issue but loopback detection is on the ports for host 1 and that works with no issue. I have been doing extra reading on the different teaming modes all day but haven’t been able to figure anything out.
Sorry for long post. Anyone have any idea what this might be or has anyone seen anything like this before? Appreciate your help and thank you for your time.
