VMHBA Sanity Check

[Jwoods]

Running esxcfg-rescan on both HBAs yields mappings on the first hba, but nothing on the second hba. This is because esx isn't balancing across both hbas...correct?

Now once I manually...errr, scriptually balance the paths across the hbas this will in fact cause IO to travel on hba1 or hba2 depending on the set preference...correct?

The real question: After preferred paths are set and balanced, should running esxcfg-rescan on both hbas yield paths on both hbas??? Currently it's not and I thought it was supposed to.

What puzzles me more is that rescan from the console doesn't show anything for the 2nd HBA, but vcenter shows paths for both hbas! It's been awhile since I've fiddled with the HBAs. Just got a set of x3950s and now setting up the fibre.

SAN: DS8300

Message was edited by:

Jwoods

[RParker]

You should have mappings attached to your HBA's. You can map an HBA (which are uniquie identifiers) to a LUN, for failover.

But in our case I only use 1 port on a dual port HBA, so when I scan I see only 1 path (on port 1) and nothing on port 2, as it should be, because I don't have the SAN configured to be accessible to both ports.

So I would say no you would not see path's on both , and I don't think ESX does load balancing, it's how you have the multi path setup.

[Jae_Ellers]

What's the output of esxcfg-mpath -l and esxcfg-vmhbadevs -q?

[Jwoods]

I don't think ESX does load balancing, it's how

you have the multi path setup.

This I'm aware of...

I'm using a pair of single port ql2460's, which I assume should show me paths on both hbas.

[Jwoods]

What's the output of esxcfg-mpath -l and

esxcfg-vmhbadevs -q?

esxcfg-rescan vmhba1 & 2:[/b]

\[nimda@chiesx01 root]# esxcfg-rescan vmhba1

Rescanning vmhba1...done.

On scsi1, removing: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.

On scsi1, adding: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.

\[nimda@chiesx01 root]# esxcfg-rescan vmhba2

Rescanning vmhba2...done.

On scsi2, removing:.

On scsi2, adding:.

\[nimda@chiesx01 root]#[/code]

esxcfg-mpath -l:[/b]

Disk vmhba1:0:0 /dev/sda (14336MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:0 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:0 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:0 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:0 On

Disk vmhba1:0:1 /dev/sdb (51200MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:1 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:1 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:1 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:1 On

Disk vmhba1:0:10 /dev/sdc (307200MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:10 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:10 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:10 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:10 On

Disk vmhba1:0:11 /dev/sdd (307200MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:11 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:11 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:11 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:11 On

Disk vmhba1:0:12 /dev/sde (614400MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:12 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:12 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:12 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:12 On

Disk vmhba1:0:13 /dev/sdf (614400MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:13 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:13 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:13 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:13 On

Disk vmhba1:0:14 /dev/sdg (307200MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:14 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:14 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:14 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:14 On

Disk vmhba1:0:15 /dev/sdh (614400MB) has 4 paths and policy of Fixed

FC 2:1.0 210000e08b9142c9<->50050768011026e6 vmhba1:0:15 On active preferred

FC 2:1.0 210000e08b9142c9<->5005076801102534 vmhba1:1:15 On

FC 6:1.0 210000e08b91f6c4<->50050768012026e6 vmhba2:0:15 On

FC 6:1.0 210000e08b91f6c4<->5005076801202534 vmhba2:1:15 On

esxcfg-vmhbadevs -q[/b]

vmhba1:0:0 /dev/sda

vmhba1:0:1 /dev/sdb

vmhba1:0:2 /dev/sdi

vmhba1:0:3 /dev/sdj

vmhba1:0:4 /dev/sdk

vmhba1:0:5 /dev/sdl

vmhba1:0:6 /dev/sdm

vmhba1:0:7 /dev/sdn

vmhba1:0:8 /dev/sdo

vmhba1:0:9 /dev/sdp

vmhba1:0:10 /dev/sdc

vmhba1:0:11 /dev/sdd

vmhba1:0:12 /dev/sde

vmhba1:0:13 /dev/sdf

vmhba1:0:14 /dev/sdg

vmhba1:0:15 /dev/sdh

[boydd]

The output looks the way it should be. Nice to see that you cut back the default paths to 4 per LUN. You should be all set.

DB

[Jwoods]

Nice to see that you cut back the default paths to 4 per LUN.

You should be all set.

Twas 8 at one point! Would've came way too close to the path max. Didn't need that many anyways.

The output looks the way it should be.

Shouldn't the esxcfg-vmhbadevs -q[/b] command give output for both vmhba1 & vmhba2.

Also, shouldn't rescan display something similiar to the following?

\[nimda@chiesx01 root]# esxcfg-rescan vmhba1

Rescanning vmhba1...done.

On scsi1, removing: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.

On scsi1, adding: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.

\[nimda@chiesx01 root]# esxcfg-rescan vmhba2

Rescanning vmhba2...done.

On scsi2, removing: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.

On scsi2, adding: 0:0 0:1 0:10 0:11 0:12 0:13 0:14 0:15 0:2 0:3 0:4 0:5 0:6 0:7 0:8 0:9.[/code]

[BUGCHK]

This is because esx isn't balancing across both hbas...correct?

No, you have become a victim of the 'canonical name' (sometimes also called the 'canonical path') feature. The multipath layer takes ownership of all paths and presents one path (the canonical one) to the upper layer.

The 'esxcfg-mpath' program goes below the multipath layer, 'esxcfg-rescan' seems to be inconsistent.

[Jwoods]

So the rescan is more a highlevel scan? And what about esxcfg-vmhbadevs? It only displays the first hba as well. Same issue...above multipath layer? Just trying to wrap my brain around this.

[BUGCHK · Accepted solution]

No, the scan is low-level - directly on the adapter, but the output seems to be high-level (above the multipath layer). I works the same way in ESX V2.5, by the way and I was confused as well, when I started with it.

And what about esxcfg-vmhbadevs? It only displays the first hba as well.

It displays the 'canonical name/path', not necessarily the 'first' path.

That makes sense, because the Service Console and its devices like '/dev/sdc' live above the multipath layer, too.