Lost all network access after update to 5.5 U3

[Eric4381]

I have an HP C7000 enclosure with 8 blades in it. Using VUM I updated one of the blades to 5.5 U3 along with other patches. This blade will no longer connect to vCenter and I have found that this blade will no longer ping out to the network also.

In DCUI I can see it still has the same IP and gateways as it did before and it looks like the NICs are still attached but I cannot get to the network at all.

Can anyone help with this? Do I need to go back to 5.5 U1?

[mars0077]

Hey there,

I would start by ruling out the hardware first and then the software.

Hardware:

1. What type of switches are you using for the c7000? Virtual connects, Cisco?

2. If you are using VC's, is your server profile correct and assigned to the bay number the blade in question belongs to? Assuming nothing has changed for the shared uplink sets.

3. If you are not using VC's, have you tried moving that blade to a different bay? Is the VLAN configuration correct the bay this server belongs to?

Software:

1. Where did the upgrade image come from? Is it an HP ISO? If not, it could be a driver issue.

2. Are you using trunks for all your network access or access ports?

3. You can reboot your host and revert to a previous version by pressing Shift-R when you see the progress bar. KB: Reverting to a previous version of ESXi (1033604) | VMware KB

4. You can restore your network configuration if the above doesn't work

[Eric4381]

We are using VC's an those were not modified in any way. I have not tried a new bay but I will test that next.

If I am correct I believe the Update manager took the patches from this uploaded ISO for HP as seen in the picture below.

I have tried resetting the entire network configuration and Re-IP it with no success. Rebooting the Host does nothing either.

[mars0077]

I probably would have rebooted and pressed Shift-R as mentioned in the article below, before restoring the network config. Another I would try is placing another working blade in the same bay as the one in question to see if it can talk to the network. This would rule out the bay and network config.

You can reboot your host and revert to a previous version by pressing Shift-R when you see the progress bar. KB: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10336...Reverting to a previous version of ESXi (1033604) | VMware KB

[Eric4381]

Update:

I have tried:

Another bay in the enclosure

Rebooting the host

Rolling back to HP ISO ESX 5.5 U2

Rolling back to HP ISO ESX 5.5 U1 Rollup

All have failed... I still have no ability to ping out or connect to the blade.

[Eric4381]

Is there any way this could be a driver issue? I checked and both VMNICs show up in DCUI which is what lead me to think it was indeed NOT a driver issue but I am not sure.

[mars0077]

You have compared the subnet mask, gateway, etc for this ESX to all others right? The last thing here is can this ESX host ping itself using its assigned IP address and/or loop address 127.0.0.1.

If there are no errors, you may need to check the firmware and confirm that it is of the save revision as all your other ESX blades. The other thing I would do is disable all vmnics and only work with the primary vmnic. This assumes that your shared uplink sets are allowing your desired VLAN and your server profile is using a single connection to the same VLAN.

We still need to rule out the hardware first. Have you tried using another blade on the same blade slot where the ESX host in question is. In other words, does a different blade work in the same slot? This would confirm that the chassis, switch, server profile and networ is all working. Also, have you checked the iLO Event log and IML log to see if there are any errors on the blade in question?

[Eric4381]

The host blade can ping its own local address but no the gateway. I have:

Switch it to use just one VMNIC

No uplink ports chagned

No VC settings changed

Default gateway is correct

This same blade was moved from its original bay to another to test...still no connection out.

What am I missing?

[TheVElement]

What is the build number of your vCenter, and what is the build number of the host that you updated and can no longer connect to vCenter?

[pedrocalixto]

Are you using Distributed Virtual Switch? Is your trunk configured properly? Did you set VLAN id of the management network ?

[Eric4381 · Accepted solution]

I should note that FINALLY I have a fix for this. I despise when people don't post the fix and just let the thread go unanswered so here it goes:

I noticed my blades, enclosure and ILO 3  firmware was low (2010 versions).

These are the thing I tried:

• Switch to different enclosure bays
• Re IP and test default gateways over and over
• Reinstall all different versions of ESXi to test functionality
• Check the Virtual connect Fiber Blades for issues

I noticed my blades, enclosure and ILO 3  firmware was low (2010 versions).

I was able to get a hold of an HP SPP CD ( HP Service Pack for Proliant Server) which had all the

Firmware, Updates, and Revisions need for that time period which brought my equipment Firmware to 2014. I retried the install of ESX 5.5 U3 and everything is functioning just fine!!!

[mars0077]

Hey Eric,

Glad it worked out for you. I do recall inquiring about the firmware levels so I'm wondering if the correct answer should have been split or fairly appointed.

"If there are no errors, you may need to check the firmware and confirm that it is of the save revision as all your other ESX blades. The other thing I would do is disable all vmnics and only work with the primary vmnic. This assumes that your shared uplink sets are allowing your desired VLAN and your server profile is using a single connection to the same VLAN."