How many LUNs visible by ESX 3.5

[scotty_p]

I just set up 3 ESX 3.5 hosts in a test environment.

I've got them attached to a couple Cisco 9000 fibre switches and I have 1 LUN attached to each switch.

The switches are trunked and are passing VSAN traffic fine (I tested it with a Windows box and can see both LUNs).

When I initially did a scan on my ESX storage adapters, they only found the 1.5TB LUN. I rebooted all 3 of my ESX servers and

then they all only found the 927MB LUN. Same thing after another scan.

Is there some limitation with ESX that it will only see 1 LUN at a time?

I've attached a snapshot of the physical layout in Fabric Manager to give you a better idea of how it's all attached.

Thanks for any help.

Scott

[Milton21]

vmware can see 255 Lun's (I think this is the right number)... so thats not the problem

[virtualdud3]

ESX 3.5 hosts can have up to 256 LUNs per server (LUNs 0-255).

http://www.vmware.com/pdf/vi3_35/r35/vi3_35_25_config_max.pdf

Do you have any sort of "LUN masking" configured?

You might try posting the output of:

#esxcfg-mpath -l

###############

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

[Milton21]

Make sure when you scan for storage you scan both new storage devices and VMFS volumes.

[christianZ]

Have you check your /var/log/vmkernel for any warnings/errors?

Have the luns different lun ids?

[Milton21]

My guess is had the right idea with the masking/zoning on the fiber controller.

[scotty_p]

Thanks for the suggestions.

I don't have LUN Masking set up and I have scanned for New VMFS and New Storage Devices. Still only the 1 LUN.

Here is the output of esxcfg-mpath -l

Disk vmhba6:0:0 /dev/cciss/c0d0 (629775MB) has 1 paths and policy of Fixed

Local 27:1.0 vmhba6:0:0 On active preferred

Disk vmhba0:0:0 /dev/sda (949471MB) has 2 paths and policy of Fixed

FC 7:1.0 10000000c9512bc4<->50060482bc007254 vmhba0:0:0 On active preferred

FC 7:1.0 10000000c9512bc4<->50060482bc007250 vmhba0:1:0 On

It only shows the 949MB LUN.

[Milton21]

how many fiber cards do you have in your ESX servers?

[scotty_p]

Just 1 card with 2 ports. I'm only using 1 port though.

[scotty_p]

I have attached the output of /var/log/vmkernel.

Does anyone think there's anything suspicious looking in there?

Thanks.

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.102 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba0, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=2, rv=0, sts=0

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.104 cpu2:1039)ScsiScan: 395: Path 'vmhba0:C0:T0:L0': Vendor: 'EMC ' Model: 'SYMMETRIX ' Rev: '5267'

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.104 cpu2:1039)ScsiScan: 396: Type: 0x0, ANSI rev: 2

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.105 cpu2:1039)WARNING: ScsiUid: 550: Path 'vmhba0:C0:T0:L0' : supports ANSI version 'SCSI-2' (0x2). In order to be used with ESX a device must support the SCSI 3 protocol.

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.105 cpu2:1039)ScsiScan: 516: Path 'vmhba0:C0:T0:L0': No standard UID: Failure

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.107 cpu2:1039)ScsiScan: 395: Path 'vmhba0:C0:T1:L0': Vendor: 'EMC ' Model: 'SYMMETRIX ' Rev: '5267'

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.107 cpu2:1039)ScsiScan: 396: Type: 0x0, ANSI rev: 2

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.108 cpu2:1039)WARNING: ScsiUid: 550: Path 'vmhba0:C0:T1:L0' : supports ANSI version 'SCSI-2' (0x2). In order to be used with ESX a device must support the SCSI 3 protocol.

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.108 cpu2:1039)ScsiScan: 516: Path 'vmhba0:C0:T1:L0': No standard UID: Failure

Jan 7 07:19:04 esx03 vmkernel: 0:00:39:35.113 cpu4:1040)SCSI: 861: GetInfo for adapter vmhba0, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=2, rv=0, sts=0

Jan 7 07:19:06 esx03 vmkernel: 0:00:39:37.161 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba0, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=2, rv=0, sts=0

Jan 7 07:19:06 esx03 vmkernel: 0:00:39:37.162 cpu2:1039)VMWARE SCSI Id: Symmetrix device with a Symm6 device id

Jan 7 07:19:06 esx03 vmkernel: VMWARE SCSI Id: Id for vmhba0:C0:T0:L0 0x01 0x83 0x50 0x12 0x57 0x00 0x00 0x00 0x00 0x53 0x59 0x4d 0x4d 0x45 0x54

Jan 7 07:19:06 esx03 vmkernel: 0:00:39:37.163 cpu2:1039)VMWARE SCSI Id: Symmetrix device with a Symm6 device id

Jan 7 07:19:06 esx03 vmkernel: VMWARE SCSI Id: Id for vmhba0:C0:T1:L0 0x01 0x83 0x50 0x12 0x57 0x00 0x00 0x00 0x00 0x53 0x59 0x4d 0x4d 0x45 0x54

Jan 7 07:19:06 esx03 vmkernel: 0:00:39:37.417 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba1, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=0, rv=0, sts=0

Jan 7 07:19:06 esx03 vmkernel: 0:00:39:37.435 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba1, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=0, rv=0, sts=0

Jan 7 07:19:08 esx03 vmkernel: 0:00:39:39.521 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba1, , max_vports=127, vports_inuse=0, linktype=0, state=0, failreason=0, rv=0, sts=0

Jan 7 07:19:09 esx03 vmkernel: 0:00:39:39.746 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba2, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:09 esx03 vmkernel: 0:00:39:39.748 cpu4:1040)SCSI: 861: GetInfo for adapter vmhba2, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:16 esx03 vmkernel: 0:00:39:47.254 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba2, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:16 esx03 vmkernel: 0:00:39:47.474 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba3, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:16 esx03 vmkernel: 0:00:39:47.477 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba3, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:24 esx03 vmkernel: 0:00:39:54.983 cpu4:1040)SCSI: 861: GetInfo for adapter vmhba3, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:24 esx03 vmkernel: 0:00:39:55.204 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba4, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:24 esx03 vmkernel: 0:00:39:55.207 cpu4:1040)SCSI: 861: GetInfo for adapter vmhba4, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:32 esx03 vmkernel: 0:00:40:02.901 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba4, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:32 esx03 vmkernel: 0:00:40:03.121 cpu2:1039)SCSI: 861: GetInfo for adapter vmhba5, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:32 esx03 vmkernel: 0:00:40:03.123 cpu7:1041)SCSI: 861: GetInfo for adapter vmhba5, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:19:40 esx03 vmkernel: 0:00:40:10.818 cpu4:1040)SCSI: 861: GetInfo for adapter vmhba5, , max_vports=0, vports_inuse=0, linktype=0, state=0, failreason=0, rv=-1, sts=bad001f

Jan 7 07:23:54 esx03 vmkernel: 0:00:44:24.838 cpu0:1024)Host: 2655: irq 6 vector 0x41 (host 0x59)

[Milton21]

what storage are you using.

[scotty_p]

An EMC Symmetrix 3830.

I just made a discovery. The line from esxcfg-mpath -l

FC 7:1.0 10000000c9512bc4<->50060482bc007254 vmhba0:0:0 On active preferred

FC 7:1.0 10000000c9512bc4<->50060482bc007250 vmhba0:1:0 On

appears like this is a redundant path to the same LUN, however these WWN's, 50060482bc007254 and 50060482bc007250 each

go to separate LUNs.

I went into the configuration of the storage adapters > Manage Paths and changed the 2nd device to active and now I only see the 1.5TB LUN.

This seems like it thinks this is a redundant path, but it's not. Does anyone know how I can get ESX to see these separately?

Thanks.

[christianZ · Accepted solution]

As I guessed you used here the same lun id (0, vmhba0:0:lun id)

>FC 7:1.0 10000000c9512bc4<->50060482bc007254 vmhba0:0:0 On active preferred

>FC 7:1.0 10000000c9512bc4<->50060482bc007250 vmhba0:1:0 On

Change it to a different one.

[scotty_p]

You were right. I changed the LUN ID and now I can see the 2nd LUN.

Here's the output from esxcfg-mpath -l

Disk vmhba6:0:0 /dev/cciss/c0d0 (629775MB) has 1 paths and policy of Fixed

Local 27:1.0 vmhba6:0:0 On active preferred

Disk vmhba0:1:0 /dev/sdb (1536418MB) has 1 paths and policy of Fixed

FC 7:1.0 10000000c9512bc4<->50060482bc007250 vmhba0:1:0 On active preferred

Disk vmhba0:0:1 /dev/sda (949471MB) has 1 paths and policy of Fixed

FC 7:1.0 10000000c9512bc4<->50060482bc007254 vmhba0:0:1 On active preferred

Thanks a lot,

Scott