NPIV, HP c-Class blades and EVA 8000

[bradmarshall-cq]

Hi all,

I'm having some issues with setting up NPIV. I've recently upgraded all my ESX servers to 3.5, and I can get a test VM to generate WWNs as per the documentation. The server believes the HBAs have NPIV enabled, as follows:

1. cat /proc/scsi/qla2300/1 | grep NPIV

NPIV Supported : Yes

Our servers are connected to a Brocade 4/24 SAN Switch, and checking the config for each port shows that NPIV is enabled.

b87f2fcsw06:admin> switchshow

...

Area Port Media Speed State Proto

=====================================

6 6 cu N4 Online F-Port 1 NPIV public

b87f2fcsw06:admin> portshow 6

portName:

portHealth: No Fabric Watch License

Authentication: None

portDisableReason: None

portCFlags: 0x1

portFlags: 0x24b03 PRESENT ACTIVE F_PORT G_PORT NPIV LOGICAL_ONLINE LOGIN NOELP LED ACCEPT FLOGI

portType: 11.0

POD Port: Port is licensed

portState: 1 Online

portPhys: 6 In_Sync

portScn: 32 F_Port

port generation number: 60

portId: 380600

portIfId: 43020012

portWwn: 20:06:00:05:1e:05:69:b7

portWwn of device(s) connected:

50:01:43:80:00:82:39:a0

Distance: normal

portSpeed: N4Gbps

However, when I boot it up I see the following (among other errors) in the vmkernel log file on the server:

Jun 16 16:25:11 b87e02esx04 vmkernel: 4:05:35:58.540 cpu1:1100)WARNING: SCSI: 745: Failed to Create vport for world 1101, vmhba2, rescan failed, status=bad0001

Which, aiui, means that the VM can't see the LUN via the SAN. I understand that I need to zone the host and then present the LUN to it, but the WWN never appears on the SAN for me to zone it. What is the correct procedure after generating a WWN on a VM, so that it will show up on the switches? Have I missed any steps in the process? Am I right in assuming this just won't work until I see the WWN of the VM turning up in the "portWwn of device(s) connected" section of the portshow? If so, how do I make this work?

Thanks,

Brad

[VirtualKenneth]

Hi,

I'm having the same issues in the same environment (only a EVA4 K instead). I still have troubles understanding the NPIV concept.

I created a VM with an RDM, from that point on I can create NPIV WWN's. NPIV support is enabled in the switches and the HBA's are OK as well:

Jun 19 14:56:10 ESX01 vmkernel: 0:03:22:14.231 cpu1:1112)<6>scsi(17) : Virtual port Created on host=0 with ID=1

Jun 19 14:56:10 ESX01 vmkernel: 0:03:22:14.231 cpu1:1112)WARNING: SCSI: 711: Created vport for world 1113, vmhba1, rv 0

Jun 19 14:56:15 ESX01 vmkernel: 0:03:22:19.233 cpu1:1112)SCSI: 623: NPIV vport rescan complete, (0x3f0eec00)

status=0xbad0001

Jun 19 14:56:15 ESX01 vmkernel: 0:03:22:19.233 cpu1:1112)SCSI: 623: NPIV vport rescan complete, (0x3f0f1680)

status=0xbad0001

Jun 19 14:56:15 ESX01 vmkernel: 0:03:22:19.233 cpu1:1112)WARNING: SCSI: 745: Failed to Create vport for world 1113, vmhba1,

rescan failed, status=bad0001

[BUGCHK]

Looks like the "hen and egg" problem...

The driver turns down the NPIV connection when it can't find any LUNs. To find any LUNs, you must have set up FC switch zoning and storage array LUN masking. To set up zoning/masking, you usually refer to the WWNs available in the switch/array user interface. They are not there because the NPIV connection was turned down...

So I've looked into the VM's .vmx file, took the WWNs from it and manually entered them into the switch zones and the array LUN masking.

Honestly, the current implementation looks like a joke for me and is absolutely not production ready. The only excuse I can accept is that this is version 0.01 and the limitations (LUN presentation to VMkernal and RDM prerequisite, NPIV turndown when no LUN can be found, ...?) will be removed in the future.

[bradmarshall-cq]

Thats what I thought was happening, the NPIV not turning on while not seeing a LUN presented to it. I'll look into seeing if I can manually enter the WWN into the zoning. The way we usually do zoning on the HP EVAs is to use the WWNs seen on the switches, so I haven't had to manually enter them before. The implementation does seem to be fairly early days at the moment - the documentation does seem a little lacking around this whole process.

--

Brad Marshall

Senior Systems Engineer

Central Queensland University

[BUGCHK]

You can set up the switch zoning and then power on the VM. I am not 100% sure as I did no thorough tests, but if I remember correctly, NPIV does enough to register on the EVA so that you can use the pull-down menu and assign the discovered WWNs.

On the other hand - a VM can have up to 4 WWPNs and as far as I can tell, it is not documented when and how these WWPNs get assigned or re-assigned to a physical path, so I put all four in every fabric and on the EVA's host entry for the VM.

[BUGCHK]

(System error caused duplicate post, sigh)

[bradmarshall-cq]

Interesting, I'm not seeing that. When looking at the switch I can only see the WWN of the ESX host, and not the VM. I had checked all of the switches and none of them had the VM's WWN show up. I'm sure I had checked our fabrics in the zoning section and couldn't see the WWNs there either, but I'll have another look. I might have to figure out how to manually enter the WWNs into the zoning.

Brad.

--

Brad Marshall

Senior Systems Engineer

Central Queensland University

[BUGCHK · Accepted solution]

A simple "switchShow" will not displaythe WWNs..

I might have to figure out how to manually enter the WWNs into the zoningGrab the values from the .vmx and then enter them through the CLI, e.g.

zonecreate "VM1_NPIV","wwn1;wwn2;wwn3;wwn4;EVA"

cfgadd "config-name","VM1_NPIV"

enable configuration

[VirtualKenneth]

I get to see the NPIV WWN in the FC switch for about 30 seconds when a VM is powered on.

After that 30 seconds the VM apparently notice that it doesn't "see" anything and disabled the NPIV again.

So, power on the VM, instantly refresh the FC switch and it should appear (at least that is the case in my test setup)

Anyhow I'm not happy with this behaviour and haven't tested NPIV any further. (haven;t manually edit zoning and EVA Hosts)

Cheers,

Kenneth

[bradmarshall-cq]

As mentioned above, the answer is to pre-zone the VM - once it boots and can see LUNs it will create the vport and all will be good.

Thanks to everyone who responded.

--

Brad Marshall

Senior Systems Engineer

Central Queensland University

[RBurns-WIS]

Has anyways actually verified that NPIV is functioning properly?

In my test with Qlogic and Emulex HBA's I found some differences in behaviour.

QLogic HBA's will briefly log into the switch, but then log out as soon as the HBA finds no LUNs available to it.

Emulex HBA's will log into the switch, and remain logged in regardless if there is presented LUNs or not.

I found this out the hard way. Though my Emulex remained logged in, my Storage processor could not see the virtual WWN of the VM. This is a behaviour of NPIV and virtual WWN's I'm told. You have to manually register the virtual WWN's in your storage system first, ensure both your ESX phsycial HBA WWN and the Virtual WWN is zoned with your Storage and only then will your Virtual WWN of the VM log into the Storage system.

I'm also told that If a VM can't access a LUN via it's virtual WWN, it will fallback to using the WWN of the phsycial adapter,

**This left me with a question. How do you know if an NPIV enable VM accessing a LUN is "really" using the virtual WWN or not??

I did a sniff on the FC traffic and found that ALL access to the LUN on my VM was coming from the FCID / WWN of the phsyical host's HBA.

Anyone see this??