VMware Cloud Community
manfriday
Enthusiast
Enthusiast

FC HBA's Losing connectivy after updating

Hi,

So last Friday I applied patch ESXi410-201107401-BG via the update manager.

This brought my ESXi hosts up to build 433742.

Since then, I have been periodically losing HBA paths.

My servers are HP DL580 G5's, with ISP2532 8gb HBA's connected to Compellent FC storage.

Everything had been running fine for the past couple years. The patch I installed is the only change that I made.

So, if I look in configuration->Storage adapters_>vmhba2 I can see the WWN of the adapter that fails.

It is something like 50:01:43:etc.. etc.

All of the ISP2532 WWN's start with a 50:01..

When the HBA dropps connection, i still see this same WWN via the VI Client.

However, if I ssh into my esxi box and do a

cat /proc/iscsi/qla2xxx/6

I can see that the WWN has changed to 21:00:00 etc...

I am not sure where this WWN is coming from.

Rebooting the host restores connectivity and the card assumes the correct WWN.

Any ideas?

Thanks guys

Jason

Reply
0 Kudos
11 Replies
JohnLambie
Contributor
Contributor

Jason,

We are experiencing the exact same issue as you. We are running IBM x3580's with Qlogic 2562 HBA's and upgraded to 433742 a couple weeks ago. Since then our Host's sporadically lose their connection to the LUN's (they show up as "inactive" in vCenter). If we happen to be at work when it happens we can put the machine in maintenance mode and rescan the LUN's and they come back right away. If not, the guests slowly drop off and the only solution is to hard boot the Host. We have opened a ticket with VMware, nothing so far except "make sure the HBA's firmware is the latest". We are in the process of upgraded the firmware right now...I'll let you know what becomes of the open ticket.

John

Reply
0 Kudos
RParker
Immortal
Immortal

So, if I look in configuration->Storage adapters_>vmhba2 I can see the WWN of the adapter that fails.

It is something like 50:01:43:etc.. etc.

All of the ISP2532 WWN's start with a 50:01..

When the HBA dropps connection, i still see this same WWN via the VI Client.

However, if I ssh into my esxi box and do a

cat /proc/iscsi/qla2xxx/6

I can see that the WWN has changed to 21:00:00 etc...

I am not sure where this WWN is coming from.

Rebooting the host restores connectivity and the card assumes the correct WWN.

Any ideas?

WWN is assigned by vendor.  Emulex starts with 10:00, Qlogic starts with 21:00, and Brocade starts with 28:00.  If you have Qlogic, assuming the qla2xxx path and 21:00 that should be the SAME WWN it's always been..

you can have Virtual nodes on a Fiber HBA, but it SHOULD be zoned as the REAL WWN.  There is a WWPN (port) and WWNN (node).  21: is the port, 20: is the node.  port is the sender and node is the destination (something close to that, I may have them reversed).

So if you are using vmhba2 that is the port assigned by VM Ware because you are using a VIRTUAL port rather than the actual adapter, that's why it's changed.  I would zone the adapter 21:00:00 and rescan the storage adapter rather than virtual, and you won't have this problem anymore.

Yes, I know it's been 2 years.. but as you see virtual ports CAN and DO change.

Reply
0 Kudos
jamesrico
Contributor
Contributor

Hi,

We've got the same problem. Running DL580 G7s with the HP branded QLogic HBA cards ISP2532

I have checked and the physical cards have WWN's starting 50:01 printed on labels.

Got a call logged with HP but I think I may have to remove the patch.

Isn't actually causing a problem though, it just causes the server to loose storage resiliancy for about 1 min every few days. The two broken paths reconnect without having to reboot the server.

Reply
0 Kudos
JohnLambie
Contributor
Contributor

We upgraded all HBA's firmware last week and it happened again early Saturday morning, one host lost connection to all it's LUN's. All guests were effected. VMware sent us a link to the latest Qlogic driver (841.k1.16.2-1vmw) which we arre applying now.

http://downloads.vmware.com/d/details/dt_esxi4x_qlogic_qla2xxx_841k12811vmw/dHdlYnQqaGhiZGhwZA==

Keep you posted.

John

Reply
0 Kudos
jamesrico
Contributor
Contributor

Great, be useful to know how you get on.

Just checked our firmware:

50014380029D62E1

Description:FC HBA Option ROM detail for QLogic HPAK344A
Version:v.2.15
Manufacturer:QLogic Corporation
Type:FC HBA Option ROM

50014380029D62E1

Description:FC HBA Firmware details for HPAK344A
Version:v. 5.02.00
Manufacturer:QLogic Corporation
Type:FC HBA Firmware

50014380029D1D81

Description:FC HBA Firmware details for HPAK344A
Version:v. 5.02.00
Manufacturer:QLogic Corporation
Type:FC HBA Firmware

50014380029D1D81

Description:FC HBA Option ROM detail for QLogic HPAK344A
Version:v.2.15
Manufacturer:QLogic Corporation
Type:FC HBA Option ROM
Reply
0 Kudos
RParker
Immortal
Immortal

A firmware update won't affect the connectivity if we are talking about a path problem on the host, 2 different issues.  One is the port reported by VM ware as 50:00 which is virtual vs the actually WWN of the HBA card, the firmware ONLY matters if you can't connect at all, it won't all of a sudden stop connecting after working for a time, that's not firmware.  You have some other problem.

Reply
0 Kudos
jamesrico
Contributor
Contributor

Our HBA cards are HP branded and have stickers on the physical cards with WWNs starting 50:00, its the same WWN as configured in HP CV and displayed in the vi client.

Reply
0 Kudos
JohnLambie
Contributor
Contributor

We don't have a pathing problem, this was a stable environment (for years) UNTIL we updated to patch 433732.

Reply
0 Kudos
JohnLambie
Contributor
Contributor

It's been 9 days since we updated the Qlogic drivers to 841.k1.16.2-1vmw and we've had no disconnects since. I've closed our ticket with VMware and hopefully this will solve the problem for good..

John

Reply
0 Kudos
jamesrico
Contributor
Contributor

Interesting, thanks John.

I rolled our servers back to the previous build (381591) on Saturday using the very useful shift-r option (http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic=/com.vmware.vsphere.upgrade.doc_40/vhuu...)

All servers have been fine since.

I will probably leave it another week then re-apply the patch and use the QLogic update.

James

Reply
0 Kudos
JohnLambie
Contributor
Contributor

Good luck to you James.. I’ll let you know if we lose our LUN’s again.

John

Reply
0 Kudos