HA explanation, and help determining what's going wrong...

[barrickd]

Trying to figure out some strange behavior with our ESX 3.0.1 hosts and VCenter 2.0.1 HA.

HA is configured on our cluster, all default settings.

What gets pinged and by who to determine if a host is isolated? Does each VM on a host ping the ServConsole Gateway or is the host doing the pinging?

Why would a host appear to be isolated in the event of router maintenance? Especially when both the service console vswitch and the vm data network vswitch are teamed with two separate physical nics, each nic connected to separate routers. And only one router is getting maintenance?

If a host is deemed isolated, and vm's are powered down by HA. They are then migrated to other hosts, but not powered back up???

[MBrownHenn]

Erik, Great recommendation to team the console NIC's and make them redundant. Although VMware puts this in their admin guide as a best practice, I am having a hard time finding documentation on how to set that up. The teaming could be simply through the hardware, but I was wondering if I could have a secondary console setup over the vmotion network or the VM network NIC's rather than installing a 7th NIC in the server (we have 4 NIC's for VM's, 1 for VMotion and 1 for the console). thoughts?

[MikeAvery]

HennITLS

Yes, you can use the same physical interfaces for any network including console and vmotion network/interfaces. How you do this depends on your network configuration.

If your switches support Q-Tagging, you can use ESX VST features to create port groups.

Let's say that you have 4 physical NICs and you divide those (2 ports each) to Q-tagged trunk ports on two different switches.

vswitch0

--> assign NIC0

--> assign NIC2

--> assign NIC3

--> assign NIC4

Under the properties dialog for vswitch0, go to the NIC Teaming tab:

Route based on: IP Hash

Network Failover Detection: Beacon probing

Notify switches: Yes

Rolling failover: No

Active Adapters:

Nic0

Nic1

Nic2

Nic3

Standy Adapters:

Unused Adapters:

Now, create a port group with matching policy exception choices for each VLAN that you might want to have a guest reside in.

Check your Console and Vmotion port groups and be sure that they are assigned to the VLAN port group (which you just created) for that specific IP subnet.

Rinse and repeat for any virtual port groups you may want to create. In this example, we have quad-redundant access to both the console and vmotion interfaces because the ports are trunked.

If you do NOT have a switch that you can configure for Q-Tagging (and the VLAN configuration of course), you need to remember that you need to associate an IP network to a physical board. You can assign port groups against those interfaces, and service consoles or vmotion interfaces against those port groups in much the same way... There is one major caveat with this approach:

You CANNOT load balance across physical switches. If you are connecting to two or more physical external switches within a "team" in your configuration, you must not attempt to load balance across two switches, it will not work. You will create Spanning Tree loops, and your server will be brought off line arbitrarily. (OK not arbitrarily, precisely as STP is designed, but if you don't know how it works it will seem random to you).

In this second case, use teaming, but with No load balancing. Failover will still work, just assign the required interfaces to run in standby mode.

Another Note, be sure to disable Spanning Tree per VLAN for any port connecting to an ESX VST port. Enable Port Fast.

Good Luck

On this switch

Message was edited by:

MikeAvery