HP 410 Array controler not working after upgrade to 4.1

[vcadmin]

Hi we are running HP 401 pci array controlers in all of our remote servers HP DL 370 G6 when we try to upgrade from ESX 4.0 to 4.1 or 4.2 when the server comes back up if freezes on the controler. We have tried various ideas here but nothing seems to work. So if anyone else has had this problem and found a solution it would be greatly appreciated for any feedback..

Regards,

Rich G

[illvilja]

Hi!

Do you have the latest firmware on the controller?

_________________________________________

VCP3/VCP4/VTSP/VSP

MCTS: Virtualization

NetApp Certified Data Management Administrator

NetApp Accredited Storage Architect Professional

[vcadmin]

Hey thanks for the help we are running the latest firmware on the hardware.. I guess my return question is are you running similar hardware and not having a problem??

Thanks richg

[DSTAVERT]

Welcome to the Communities.

The P410 is a supported controller. A quick search of the forums didn't find anything similar. I would double check that the controller firmware is up to date. This isn't a normal or common issue and I would place a call to VMware support and or HP support.

[vcadmin]

Thank you once again for the feedback. I believe my man in the UK is in contact with HP and vmware I was trying to see if anyone on the site had dealt with this problem and had a work around.. I have found more answers here for a lot of my problems in the past. The strange thing about this problem is we are having it on all our servers running this card and yes I agree with you that it is in the HCL. I will get an exact def of the problem from my man in the UK and detail here so you can have a review..

thanks again..

Richg

[DSTAVERT]

The only real recurring issue with any of the HP controllers is performance without the BBWC module. That problem isn't unique to HP as all RAID cards suffer the same issue but I mention it as the only frequently reported problem. It would be good to understand what is happening.

[vcadmin]

Thank you for your feedback, I am sorry I did not post the exact problem I am in the US and my co-worker is in the UK as soon as I speak to him tonight or tomorrow morning I will post the exact problem he is having. Thank you for posting it is greatly appreciated..

Regards,

RichG

[DSTAVERT]

More than welcome. Hopefully there is an easily solvable issue.

[vcadmin]

Here is the details of the problem we are having, this is the information I got this morning from my Admin in the UK..

So far he has tried the below with Three different servers and three different P410 controllers (and upgraded firmware to latest v3.66).

Hardware setup:

DL370G6 (Std Spec from HP)

2 x 72GB Mirrors connected to P410i controller (containing ESX and SC)

1 x Raid 5 Disk group connected to P410 PCI Controller (Guests)

Upgrade

Install:

Esxupdate -- bundle =<pre-update.zip>update

Esxupdate--bundle=>upgrade.zip>

Reboot System, will start to boot and gets to the point of loading "storage-drivers". At this point the system will halt and not go any further. (shutting the system down and removing the P410 PCi controller, boot again and it will boot fine).

Clean Install:

If you try a clean install with the same hardware setup. The setup will ask if you want to install any custom drivers (There are NO custom drivers for the P410, and it uses the in-built driver with ESX). click no and tell it to install. It will start to install drivers, and will get to the point of installing "Storage-Drivers" and Halt and the install will get no further.

If you open the server and watch the array controller when you are booting after an upgrade or running a clean install you will notice that it has the usual green lights flashing until the point of loading "Storage-Drivers", At this pint an AMBER light will be displayed showing a diagnostic error. On following boots the array controller will display that is had errors on last boot. Booting v4 again will work fine and boots cleanly.

At the moment, although I am getting daily updates from HP/Vmware we don't seem to be going anywhere. Hopefully you can press your guys over there in the USA for answers.

Any help from you folks will be greatly appreciated as we have over 15 servers dispatched around the world in remote offices running the above configuration, and need to roll out an upgrade via a remote install so removing and re-adding hardware is not an option as we can not get hands on in most of these sites..

Thanks very much.

RichG

[richG]

Hi all,

I am very glad to say this problem has been resolved. We found the problem was the on board control was using an older firmware verson while the 410 was using the latest version. This caused a conflict.. Once both cards were on the same version the problem was resolved. So this problem can be considered closed.

Thanks for everyones feeback, it was greatly appreciated.

Regards,

RichG

[Josh26]

Hi,

If I read you correctly, you have an integrated P410i running 2 x HDD's, and another RAID card, a PCI P410 running the rest of your HDD's ?

I can't see why you'd do this, and it's probably why noone has run into your issue before. The P410i integrated on that board is more than capable of running a fully loaded bay worth of disks (as long as you purchase the BBWC). The second card only complicates matters.

Edit: Two full disks for an ESX service console seems overkill. On a side issue, if you are doing fresh deployments, why not ESXi?

[richG]

Hi to answer your question regarding the reason we use the onboard arrary controller for our internal drives and another card for the other drives the short answer it company policy not my call. As I am sure there are certain things each of us have to do that we might question but to keep our jobs we follow the request of the management this is my case.  As far as not moving to ESXi we feel there is no need at this point in time to make the move..

Thanks RichG

[vcadmin]

The guests are homed on alternate array controllers for several reasons, most of which are pretty good –

- Improved PCIe bus throughput: parking guest arrays on additional array controllers allows guests a dedicated PCIe bus channel as opposed to sharing bus requests with the host OS, this also allows you to set bus priority via card placement, if you’re homing production and QA VMs on the same host for example you can park the production controller and array on a PCIe bus with a higher priority and/or wider throughput channel (if using an 8x AC like the P812) to give production guest PCIe bus requests a higher queue priority at the HW level.

- Increased internal volume size: Using 8 drive dual or single channel SAS drive cages on an HP DL380 or DL370 platform you’d be limited to creating either a 6 drive RAID1 guest volume or two 3 drive RAID5 guest volumes were you to share a cage and onboard controller with the VS4 OS volume as that volume should be a 2 drive SP mode RAID1 set split across channels for redundancy. You’re either going to be lacking IOPS or capacity in going with 3 drive RAID5 sets with 146GB still being the capacity ceiling for HP SFF SAS 15K drives, a 6 drive RAID1 set would provide sufficient IOPS using 15K drives but would provide even less capacity net new over the smaller pair of RAID5 arrays. Using all 8 drives in a cage in DP mode at RAID5 will give you both acceptable IOPS throughput and capacity.

- Faster failed array controller recovery: Swapping a failed PCIe slot based controller out is a bit less painful than digging the board based unit out, HP has improved this in recent models, we’re at least out of the stone ages where losing an onboard array controller meant replacing the whole of the system board.

The above may very well be overkill for some shops, but unfortunately under-kill can get you killed in certain cases.

Good luck out there kids,

J.

[DSTAVERT]

As most SAS server drives are dual port it requires two controllers to provide for that redundancy.