ESXi 5.1 management network no connectivity with two nics connected

[TDInc]

IBM H Chassis populated with HS22 blades

Blades use Broadcom 5709S dual gigabit NIC

With ESXi 5.0, system was running fine with both NICs connected (with static IP) in the management network settings (able to ping in and out). Performed upgrade to ESXi 5.1 preserving VMFS datastore. After reboot, management network shows both NICs connected with static IP still assigned but I haven't any network connectivity (unable to ping in our out). If I disable any one of the NICs, and restart the management network, I can immediately ping in and out.

I have taken a spare blade and tested this by upgrading from 5.0 to 5.1 and installing 5.1 clean, both producing same results. A 5.0 host with both NICs enabled (whether static IP or DHCP) can ping in and out. A 5.1 host with both NICs enabled (whether static IP or DHCP) cannot ping in our out. 5.1 host cannot obtain DHCP address, but fails back to APIPA.

[fdapra]

Is the management network configured on a different network from your production environment?

[TDInc]

No. Network configuration between 5.0 and 5.1 has not changed.

[TDInc]

The IBM BladeCenter H chassis has two BNT Layer 2/3 Modules. The HS22 has two gigabit network adapters (Broadcom 5709S). The way the chassis works is one adapter gets mapped to BNT module in bay 1 and the other adapter gets mapped to BNT module in bay 2. This creates a separate switch scenario, which means LACP does not work. This did not seem to be an issue in ESXi 5.0 but appears to be the break point in 5.1. Any ideas?

[a_p_]

Since you mentions LACP, how did you configure the port group/vSwitch policy. With separate switches (i.e. not stacked), you should be using the default policy (Port based) with the physical switch ports setup as either access/tagged or trunk/untagged ports.

André

[TDInc]

I do not have anything set up other than factory defaults. Take a clean blade and install ESXi 5.1. In the management network settings, connect both NICs (5709S), DHCP, reboot, and all 0's.

Do the same with a clean blade using 5.0 and all works. I do not know what VMware changed between 5.0 and 5.1 in respect to default NIC configurations for a standard vSwitch or what I need to undo.

[admin]

What is the management port group settings. Is it set to IP hash since you mentioned LACP...

[NagangoudaPatil]

Download NIC drivers for ESXi 5.0/5.1 using vMA or Power CLI install the drivers once again and verify the status.

[piaroa]

Have you tried installing the IBM custom version of ESXi 5.1, it comes with IBM drivers pre-packaged, that has solved a few issues for me in the past (installing vendor specific builds).

[TomHowarth]

did you use the standard downloadable version of 5.1 or IBM's version, if the former, try the upgrade using IBM's 5.1 ESXi build as this will include all the relevant device VIBs for the model of Host

[TDInc]

I have tried both downloading the drivers and updating as well as downloading IBM's 5.1 version with the same results. I am guessing anyone can recreate this problem on any hardware.

1. Take a machine with two identical NICs, connect each NIC to a separate, non-stacked switch.

2. Connect at least one switch to a network where a DHCP address can be obtained. (No advanced networking, VLANs, or anything else of that nature. Absolutely no configuration on the physical switches. For simplicity, only have one switch connected to a live network, other switch is just simply powered on.)

3. Perform a clean install of ESXi 5.0.

4. Do not change any values when you install, other than your install location if necessary.

5. When your ESXi installation finishes and reboots, go into the console and connect both NICs to the Management network

6. Reboot.

7. When your ESXi host comes back up, you should get a DHCP address and be able to ping the host and from the host as well as connect with the client.

Repeat steps these steps, with 5.1. Does anyone get a DCHP address or any type of communication?

[TDInc]

Additional note: In an attempt to simplify the setup and better understand this issue and its resolution, I am only using one blade and only using ESXi (no vCenter). Using the steps above, anyone should be able to recreate this issue.

[TDInc · Accepted solution]

I found the problem. For whatever reason, both switches must be actively connected on the same network for the observed networks to provide correct info for teaming.