VMware Cloud Community
touimet
Enthusiast
Enthusiast

HBA Multipathing confusion

I can’t seem to understand why all the 572235MB disks, which are only 8 LUN’s total, are not multpathing.

Here is my esxcfg-mpath –l output.....

# esxcfg-mpath -l

Disk vmhba4:0:2 /dev/sdo (371820MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->50060482c3a05c1d vmhba4:0:2 On active preferred

Disk vmhba3:0:3 /dev/sdc (371820MB) has 3 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->50060482c3a05c02 vmhba3:0:3 On active preferred

FC 22:15.0 210000123fcf52ce<->50060482c3a05c12 vmhba3:1:3 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c0d vmhba4:1:3 On

Disk vmhba4:0:3 /dev/sdp (371820MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->50060482c3a05c1d vmhba4:0:3 On active preferred

Disk vmhba3:2:10 /dev/sdm (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:10 On active preferred

Disk vmhba3:0:4 /dev/sdd (371820MB) has 3 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->50060482c3a05c02 vmhba3:0:4 On active preferred

FC 22:15.0 210000123fcf52ce<->50060482c3a05c12 vmhba3:1:4 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c0d vmhba4:1:4 On

Disk vmhba3:3:10 /dev/sdag (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:10 On active preferred

Disk vmhba4:2:10 /dev/sdy (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:10 On active preferred

Disk vmhba3:2:11 /dev/sdn (572234MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:11 On active preferred

Disk vmhba3:3:11 /dev/sdap (572234MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:11 On active preferred

Disk vmhba4:3:10 /dev/sdan (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:10 On active preferred

Disk vmhba4:2:11 /dev/sdz (572234MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:11 On active preferred

Disk vmhba4:3:11 /dev/sdao (572234MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:11 On active preferred

Disk vmhba4:0:4 /dev/sdq (371820MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->50060482c3a05c1d vmhba4:0:4 On active preferred

Disk vmhba3:2:4 /dev/sdg (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:4 On active preferred

Disk vmhba3:3:4 /dev/sdaa (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:4 On active preferred

Disk vmhba4:2:4 /dev/sds (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:4 On active preferred

Disk vmhba3:2:5 /dev/sdh (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:5 On active preferred

Disk vmhba4:3:4 /dev/sdah (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:4 On active preferred

Disk vmhba3:3:5 /dev/sdab (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:5 On active preferred

Disk vmhba4:2:5 /dev/sdt (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:5 On active preferred

Disk vmhba3:2:6 /dev/sdi (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:6 On active preferred

Disk vmhba4:3:5 /dev/sdai (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:5 On active preferred

Disk vmhba3:3:6 /dev/sdac (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:6 On active preferred

Disk vmhba4:2:6 /dev/sdu (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:6 On active preferred

Disk vmhba3:2:7 /dev/sdj (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:7 On active preferred

Disk vmhba4:3:6 /dev/sdaj (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:6 On active preferred

Disk vmhba3:3:7 /dev/sdad (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:7 On active preferred

Disk vmhba4:2:7 /dev/sdv (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:7 On active preferred

Disk vmhba3:2:8 /dev/sdk (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:8 On active preferred

Disk vmhba4:3:7 /dev/sdak (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:7 On active preferred

Disk vmhba3:3:8 /dev/sdae (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:8 On active preferred

Disk vmhba4:2:8 /dev/sdw (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:8 On active preferred

Disk vmhba3:2:9 /dev/sdl (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:9 On active preferred

Disk vmhba4:3:8 /dev/sdal (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:8 On active preferred

Disk vmhba3:3:9 /dev/sdaf (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:9 On active preferred

Disk vmhba4:2:9 /dev/sdx (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:2:9 On active preferred

Disk vmhba4:3:9 /dev/sdam (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:3:9 On active preferred

Disk vmhba3:0:10 /dev/sde (371820MB) has 3 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->50060482c3a05c02 vmhba3:0:10 On

FC 22:15.0 210000123fcf52ce<->50060482c3a05c12 vmhba3:1:10 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c0d vmhba4:1:10 On active preferred

Disk vmhba4:0:10 /dev/sdr (371820MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->50060482c3a05c1d vmhba4:0:10 On active preferred

Disk vmhba3:0:0 /dev/sdb (0MB) has 4 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->50060482c3a05c02 vmhba3:0:0 On active preferred

FC 22:15.0 210000123fcf52ce<->50060482c3a05c12 vmhba3:1:0 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c1d vmhba4:0:0 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c0d vmhba4:1:0 On

Disk vmhba0:0:0 /dev/sda (69376MB) has 1 paths and policy of Fixed

Local 12:14.0 vmhba0:0:0 On active preferred

Disk vmhba3:0:11 /dev/sdf (371820MB) has 3 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->50060482c3a05c02 vmhba3:0:11 On

FC 22:15.0 210000123fcf52ce<->50060482c3a05c12 vmhba3:1:11 On

FC 22:15.1 220000123fcf52cf<->50060482c3a05c0d vmhba4:1:11 On active preferred

0 Kudos
4 Replies
kastlr
Expert
Expert

Hi,

what kind of target[/b] are you using?

From the WWN's in your output it does looks like that you're using two QLogic HBA's and two different storage arrays.

The one in question does look to me like it uses Emulex HBA's.

Are you sure that this array doesn't require some special configuration tasks when a LUN is presented on multiple front end ports?


Hope this helps a bit.
Greetings from Germany. (CEST)
touimet
Enthusiast
Enthusiast

I have a NexSAN that is SCSI attached to 2 Crossroads. The NexSAN presents all 8 LUNS on both port0 & port1.

NexSAN Port0 --> Crossroads_A

NexSAN Port1 --> Crossroads_B

Each Crossroads has 2 FC ports that connect to 2 seperate fiber fabrics. So

NexSAN Port0 --> Crossroads_A --> 2 FC ports --> Fiber Switch --> ESX vmhba3

NexSAN Port1 --> Crossroads_B --> 2 FC ports --> Fiber Switch --> ESX vmhba4

0 Kudos
jhanekom
Virtuoso
Virtuoso

My guess would be that the LUN IDs over the various paths do not match, which is a requirement for ESX's multipathing to work correctly. That's usually the cause of this type of picture. Ironically, troubleshooting is a bit more difficult because you've (imho, correctly) standardised on a set size for your volumes.

Try un-presenting the LUNs and adding them back in one at a time. Rescan each time and verify that what's LUN x on vmhba3:0 is also LUN x on vmhba3:1, vmhba4:0 and vmhba4:1.

touimet
Enthusiast
Enthusiast

Here is how the VIC is displaying those LUN's

SCSI Target 2 (vmhba3)

Path | Canonical Path | LUN ID

vmhba3:2:4 | vmhba3:2:4 | 4

vmhba3:2:5 | vmhba3:2:5 | 5

vmhba3:2:6 | vmhba3:2:6 | 6

vmhba3:2:7 | vmhba3:2:7 | 7

vmhba3:2:8 | vmhba3:2:8 | 8

vmhba3:2:9 | vmhba3:2:9 | 9

vmhba3:2:10 | vmhba3:2:10 | 10

vmhba3:2:11 | vmhba3:2:11 | 11

SCSI Target 3 (vmhba3)

Path | Canonical Path | LUN ID

vmhba3:3:4 | vmhba3:3:4 | 4

vmhba3:3:5 | vmhba3:3:5 | 5

vmhba3:3:6 | vmhba3:3:6 | 6

vmhba3:3:7 | vmhba3:3:7 | 7

vmhba3:3:8 | vmhba3:3:8 | 8

vmhba3:3:9 | vmhba3:3:9 | 9

vmhba3:3:10 | vmhba3:3:10 | 10

vmhba3:3:11 | vmhba3:3:11 | 11

SCSI Target 2 (vmhba4)

Path | Canonical Path | LUN ID

vmhba4:2:4 | vmhba4:2:4 | 4

vmhba4:2:5 | vmhba4:2:5 | 5

vmhba4:2:6 | vmhba4:2:6 | 6

vmhba4:2:7 | vmhba4:2:7 | 7

vmhba4:2:8 | vmhba4:2:8 | 8

vmhba4:2:9 | vmhba4:2:9 | 9

vmhba4:2:10 | vmhba4:2:10 | 10

vmhba4:2:11 | vmhba4:2:11 | 11

SCSI Target 3 (vmhba4)

Path | Canonical Path | LUN ID

vmhba4:3:4 | vmhba4:3:4 | 4

vmhba4:3:5 | vmhba4:3:5 | 5

vmhba4:3:6 | vmhba4:3:6 | 6

vmhba4:3:7 | vmhba4:3:7 | 7

vmhba4:3:8 | vmhba4:3:8 | 8

vmhba4:3:9 | vmhba4:3:9 | 9

vmhba4:3:10 | vmhba4:3:10 | 10

vmhba4:3:11 | vmhba4:3:11 | 11

I "un-presented" all but 1 LUN and here is the esxcfg-mpath -l output:

Disk vmhba3:2:4 /dev/sdg (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108c vmhba3:2:4 On active preferred

Disk vmhba4:2:4 /dev/sdp (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e00203108b vmhba4:2:4 On active preferred

Disk vmhba3:3:4 /dev/sdh (572235MB) has 1 paths and policy of Fixed

FC 22:15.0 210000123fcf52ce<->100000e00203108a vmhba3:3:4 On active preferred

Disk vmhba4:3:4 /dev/sdq (572235MB) has 1 paths and policy of Fixed

FC 22:15.1 220000123fcf52cf<->100000e002031089 vmhba4:3:4 On active preferred

Now the VIC displays the following:

SCSI Target 2 (vmhba3)

Path | Canonical Path | LUN ID

vmhba3:2:4 | vmhba3:2:4 | 4

SCSI Target 3 (vmhba3)

Path | Canonical Path | LUN ID

vmhba3:3:4 | vmhba3:3:4 | 4

SCSI Target 2 (vmhba4)

Path | Canonical Path | LUN ID

vmhba4:2:4 | vmhba4:2:4 | 4

SCSI Target 3 (vmhba4)

Path | Canonical Path | LUN ID

vmhba4:3:4 | vmhba4:3:4 | 4

0 Kudos