We have the beginnings of an HA environment in place, but things aren't quite working as expected and I'm just trying to wrap my mind around why.
Below is a rough image of our network setup:
Per the VMware Networking Best Practices, I understand this is not an ideal/robust setup, but I believe it should still work. Here's the situation and where thinsg are falling apart:
On both ESX hosts we are using NIC Teaming with the Route based on the originating virtual port ID setting. Both vmnic4 and vmnic5 are Active.
VM1has been assigned to vmnic4 (per esxtop on ESX2)
VM2 has been assigned to vmnic5 (per esxtop on ESX1)
VM1 cannot reach VM2 and vice versa.
If I change vmnic4 on ESX2 to Unused (IOW, I disable it) -- VM1 gets moved to vmnic5 (on ESX2) and can then reach VM2 on ESX1.
Now, the obvious solution is to connect the two switches together either via an Ethernet cable or stacking (the prior most likely since these are Cisco 2960G switches which, as I understand it, cannot be stacked).
However, per the Networking Best Practices Document (by Jeff Hunter), the exact configuration I describe above is listed in his document as not ideal, but technically should function.
I would think that an ARP request from VM1 would pass over ESX2:vmnic4 into Cat 52 and be able to exit via ESX1:vmnic4 and find its way to VM2. Perhaps there is some sort of configuration in the vSwitch or port group that I'm missing?
We're planning to cross connect the switches as it's a more robust solution (or implement LACP), but I'd like to wrap my brain around why the current setup isn't working.
Thanks in advance!
