HP VMware ESXi 5.1 update 3 upgrade kills Emulex network and cannot be rolled back ?

[AlbertWT]

People,

I'm having some serious issue here after upgrading the HP Blade BL 465c G7 with Emulex NC551i Dual Port FlexFabric 10 Gb adapter.

The problem: After successful inline update from ESXi 5.1 Update 2 Patch 5 (2000251) into ESXi 5.1 Update 3 (2323236) the ESXi server cannot be pinged nor it can ping to outside network, therefore it failed to reconnect back to VCenter server.

Software/Firmware version:

Before the upgrade of the server the version are as follows:

~ # ethtool -i vmnic0

driver: be2net

version: 10.2.293.0

firmware-version: 4.9.416.2

bus-info: 0000:04:00.0

~ #

Since the version above doesn't work, I've managed to download VMW-ESX-5.1.0-be2net-10.2.477.20-offline_bundle-2541837.zip to upgrade the VMNic driver it into: 10.2.477.20 but still after the update and reboot the network is not ping-able.

Rollback:

I have also tried to perform rollback to the previous working ESXi server version 2000251 using Shift+R but the ESXi server still rebooting as 2323236 with no working network still.

Any other suggestion would be greatly appreciated without having to reconfigure the ESXi settings and Distributed Switch configuration from scratch.

[AlbertWT]

But for my other HP Blade BL 465c G8 with Emulex HP FlexFabric 10Gb 2-port 554FLB adapter it was upgraded successfully with no issue at all.

I'm using the same .ISO files from HP ESXi as below:

VMware-ESXi-5.1.0-Update2-2000251-HP-5.69.2-Nov2014

VMware-ESXi-5.1.0-Update3-2323236-HP-510.9.3.16-Jun2015

[Rubeck · Accepted solution]

According to the HP/VMware recipe here: http://vibsdepot.hp.com/hpq/recipes/HP-VMware-Recipe.pdf

There's a note regarding the NC551i:

/Rubeck

[AlbertWT]

Rubeck‌ yes, that's why I have download VMW-ESX-5.1.0-be2net-10.2.477.20-offline_bundle-2541837.zip and manually install it after the upgrade but still not working ?

Do I need to update the firmware as well to 4.9.416.7 ?

[AlbertWT]

Rubeck‌ Today I got the chance to perform the firmware update on the HP BL 465c G7 that is affected with this issue.

as per (HP NC551i) Emulex OneConnect OCe10102 10GbE NIC CNA for HP ProLiant AMD G7 BladeSystems page recommendation:

http://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=19068&deviceCat...

as you can see above, the ESXi 5.1 Update 3 host still un-pingable and cannot connect to the network.

VMware HCL says HP BL 465c G7 is supported in 5.1 Update 3: http://www.vmware.com/resources/compatibility/detail.php?deviceCategory=server&productid=14145&relea...

Therefore the onlyway to make it working today was to perform clean isntall of ESXi 5.1 Update 2 and configuring everything manually one by one.

[AlbertWT]

OK, I've found out that updating to the ESXi 5.1 Update 3 will upgrade & overwrite the stable driver version into

~ # ethtool -i vmnic0

driver: be2net

version: 10.5.65.4

firmware-version: 4.9.416.2

bus-info: 0000:04:00.0

~ #

hence I will need to manually install the stable driver using VMW-ESX-5.1.0-be2net-10.2.477.20-offline_bundle-2541837.zip to make sure it is within the stable recipe.

After the driver downgrade, all went to normal.

[de2rfg]

I just performed an update of an DL580G7 host with VMware Update Manger which includes the HP vibsdepot and also an SPP update with HP SUM. My NC550SFP NIC has now the following combination.

~ # ethtool -i vmnic4

driver: be2net

version: 10.5.65.4

firmware-version: 4.9.416.7

bus-info: 0000:0e:00.0

Right now everything seems to be working just fine, I can ping the host and also the VMs. But I'm a bit nervous because of the notes in the HP recipe about NC550SFP NIC. Did you update you driver/firmware combination since you last post?

[AlbertWT]

de2rfg‌ sorry for the late reply,

My G7 servers that are stable in production is now using the same firmware version as you 4.9.416.7, however the driver is on 10.2.293.0

I'm running ESXi 5.1 Update 3 not 5.5 yet. what's your environment ?

hope that helps.

[PieterjanMonten]

Not the same versions of hardware or software but comparable, the BL460C with the 553i emulex works with the service pack for proliant (SPP) 2015.6 (including the emulex firmware v. 10.5.65.21) on esxi 5.5u3 HP.