I have a cluster setup with shared storage on a Dell MD3000i. I am trying to create redundancy in my service console ports/vm network . I set up a single Vswitch which has my vm Network and Service Console attached. I have added two nics to the v switch for redundancy. I start a constant ping to both the service console IP and the IP of the vm I am running for test. When I pull nic 0 everything stays up just fine and I only have one droppped packet before the redundant physcal nic 1 takes over. Now when i plug nic 0 back in I get multiple packet drops on the service console ip and the vm guest. Then HA takes over and brings the VM on the secondary host within my cluster. Am I missing something? Why is my service console heartbeat dropping for that long when I plug the redundant nic back in?
Assuming this isn't a physical switch problem or a duplex problem (check those first) or a network cable failure, see if making the NICs a pair of Active/Standby reverse pairs for a) your Service Console and b) your VM will help the situation. Assuming this isn't a production ESX host, go into the Configuration Tab for your ESX host, look at Networking and click the Properties of your VSwitch. Highlight Service Console and click Edit. Click Continue, Go to the NIC Teaming tab -- you should see both your physical NICs here -- and click the box for "override Failover Order. Move one of the NICs to Standby and keep one of the NICs for Active. In the NIC teaming for your VMs reverse the NICs order for your Active and Standby.
See if that helps the ping situation.
You may need to check your entire networking setup there. I suspect there are looping issue. I hit to the same problem in the pass and manage to solve the looping issue.
Just a liitle more info on my setup. I have two esx 4 hosts and one md3000i for shared storage. they are all connected to a single layer 2 switch which is vlan'd out. One vlan for host network and one vlan for the iscsi network. Its a test environment to see how this all works. I did try the active and standby nic for the single vswitch, but that still did not solve my issue. i then tried setting both Nics be the active in case one dropped the other would take over, but again when I tested it out by pulling the one of the active nics waiting a couple minutes for vcenter to recognize the drop and then plugging the nic back in the connection to the host and to vmware guest would drop roughly 15 to 20 ping packets. enough for Vcenter HA to take over and fail it over to the other host.
I also went into the nic teaming options and set the fail back to no to make sure that the connection wouldn't attempt to fail back to the active connection. No Luck.
How was your configuration setup? Since this is a test environment we are plugging in both nics for the service consoles for both hosts into the same switch. Its my understanding that it shouldn't be an issue since all traffic is essentially on layer two with this setup.