ESXi 5 hangs during boot when loading be2net driver

[BjornJohansson]

Hello all,

I have a problem with all of the six new BL620c G7 we recently bought.

My problem is that ESXi 5 hangs during reboot while loading be2net driver. This problem occurs randomly during reboot only and it hangs when loading screen says "Loading module be2net..."

The problems fits into http://kb.vmware.com/kb/1033314 except that the KB only is about 4.1. No other problems, never looses contact with vSphere Client as some KB suggests.

I have updated be2net driver to HP latest version and firmware.

Blades are up to date according to the latest Smart Update Firmware DVD 9.30 except for NICs who are upgraded to:

Driver version: 4.0.355.1

Firmware version: 4.0.360.15

Onboard Administrator: 3.32

HP VC Flex-10 3.15 (a bit old, I know)

Anyone seen this? Any suggestions?

Thanks.

/B

[Paul11]

I'm using the same HW + SW but I'm using HP VC Flex-10 3.30.

I would first check the VC-Profile, if you don't have a wrong konfiguration with a iSCSI HBA.

If you are using an iSCSI HBA, it must be LOM:1-b and LOM:2-b.

If you are not using an iSCSI HBA, be sure that you have no definition in the iSCSI HBA section.

(I can't remeber the GUI in V3.15, but in V 3.30 there are separate sections for iSCSI HBA, FC HBA and FCoE HBA)

Be sure that there is no unused (or Unassigned)  adapter in this section!

If everything is O.K., I would do an update to V3.30 or maybe to the new V3.51.

Paul

[BjornJohansson]

Hello,

Sorry forgot to mention that we use NFS only on storage and a trunk to regular network of course. No iscsi adapters defined.

After reading blogs etc I get a bit hesistant to go for the new firmware, seems that some new versions contained many bugs. How is 3.30 working for you?

Thanks

[Paul11]

As far as i can remeber, V3.15 was a very bad FW for as, because there was a big problem if DNS entries where used.

So we used V3.18 as soon as possible. We have lots of trouble with the BL620c with the old NIC FW with VLAN-Tagging before

Firmware version: 4.0.360.15

Now we are using V3.30 and I think this is one of the best FW versions.

V 3.51 is very new but has some improvements in loop protection, but I will wait a few month with the update.

So V3.30 with the actual NIC-FW and drivers works very well with ESXi V5.0.

[BjornJohansson]

Ok, the VLAN problem - did you have problem with BL620c not booting? Hangning when loading be2net?

I double checked server profile. We acutally had non active FC HBA connections (no iSCSI though) but I deleted those. Unassingned the profile, assigned it again and booted. Still hangs when loading be2net... longshot that did'nt work. So it's probably VC upgrade that remains I guess.

Can you upgrade VC modules without disruptions on NICs (NFS and network)?

Thanks

[Paul11]

Yes I know this problem with BL620c not booting. You can press Alt F11 in the ILO-Console to see the error message, when it is hanging.

I have this problem after updating the NIC-FW without doing a power off. I think the problem was solved with the latest NIC FW-Update and a power off.

But we do not boot the BL620c very often, so maybe the problem will come back again...

I would also try to remove the BL620c or do a "reset server ‚bay number‘ " via the Onboard Administrator, so you can be absolut sure, the modified profile is loaded to the server.

I can't remember having this problem after a power off. Only reboots sometimes hang.

You can do the Virtual Connect FW-Update online, look at these links:

http://communities.vmware.com/thread/340218?tstart=30

http://v-front.blogspot.com/2011/12/how-to-do-online-virtual-connect.html

Be sure, you have marked  "smart link" in the Ethernet Network on the Virtual Connect module.

[sakibpavel]

I am experiencing this issue.

Sakibpavel

[BjornJohansson]

Thanks for the links. I will plan an upgrade ASAP, will post here if hang issue is resolved with new firmware. Might take some time though, I must also remove some old VC-FC modules/HBAs before upgrading. Want to remove possible factors that might genereate problems.

Anyone has any experience with firmware version 3.51? Buggy or stable as a rock?

Thanks,

[BjornJohansson]

Please note that an online update will probably affect your environment.

I did ran the update with manual reboot and activation. It is correct in the blog post that you will only lose one ping, that happend in my environment too. But when I reseted the backup Flex-10 module all VMs lost their network connection causing some minor problems. Users had to reconnect their applications.

Also, we have NFS storage so all VMs issued a time out on disk. No reports on crashed VMs though.

I will activate the other module off hours. Until then I have one module with 3.15 and one on 3.51. Not optimal but I don't want to disrupt any work.

[BjornJohansson]

I called HP Support to make sure. They sent me a matrix from VMware I've never seen before, using be2net and vSphere 5 VC firmware 3.18 is the highest supported. So Im going with this, actually downgrading...

http://partnerweb.vmware.com/programs/hcl/ESX_Flex_config.pdf

This firmware updating is a nightmare...impossible to find stuff, buggy firmware and comlex support matrixes spread out over VMware and HP site.