Hosts seeing Equallogic storage but not connecting

[FunkyD]

I have 3 hosts, one is version 4 and can see all the targets on the Equallogic PS6000 (firmware 5.1.2) . The other 2 hosts are bult with version 5 using the Dell customised install.

I have configured everything as far as I can tell correctly:

- iSCSI adapters configured with one to one mapping of vmkernal ports and physical adapters

- hosts can ping every IP on the iscsi network (dedicated switches, no routing)

- heartbeat set up without a VLAN

- No authentication on volumes (for testing)

- Jumbo frames enabled on physical switch, virtual switch and adapters (9000)

- software iSCSI adapter added and bound to iSCSI adapters

The software iSCSI adapter can see all the volumes under static discovery but will not connect to them. In the logs it says:

connect failed (113,No route to host)

Login failed: ign.....(iSCSI ID)

I've seen the patch that came out for ESX 5 and have added it to update manager but it says I don't need it which I presume is because the Dell build came out after the patch was released and therefore is included.

I can't think what else to check - any ideas?

Much appreciated

[JESX35]

If i remeber correctly you don't need to setup a static target, you use dynamic with the equallogic and just point it to the Group IP address for your iSCSI.

Once you have that setup you create Volume and under the access tab put the iqn numbers of all your hosts.  You can find the IQN number on you iSCSI hba which is usally vmhba33 but it can differ from host to host.

Once all Volumes have access to all hosts re-scan you iSCSI hba and you should have full access to the volumes.  Also don't forget to set the MPIO on the Datastores to Round Robin

[FunkyD]

I've done that but nothing shows up after the scan.

I didn't think you had to put the iqn into the access section of the Equallogic -  you can just leave it as unrestricted?

[JESX35]

You can connect it with IQN or IP address.  I usally use the IQN as it will load blance better based off you settings on the ESX host.  When you set the IP address you would have to set both the iSCSI NIC ip addresses and i just find it gets messy then.

I don't know if there is a * * kind of all access setting in the Equallogic, there might be but I usally didn't do it anyhow.  It doesn't take much to add the IQN's into the access page.  Plus then you will never have other devices that can connect to your storage.  Try setting it up with the IQN number and see.  Also make sure when you copy or paste the IQN into the access page on the volumes that there is no blank spaces at the end as this can cause it not to connect.

Also one last thing to check is to make sure CHAP is not enabled.  If you have CHAP enabled you would need the CHAP key before you would be able to connect.

So you access tab on one of your volumes should look something like this

iqn.1997-2007-esxhost01.domain.com

iqn.1997-2007-esxhost02.domain.com

ect ect

Also if you go to the console of your ESX host (If you are using ESXi you will need to connect to your hosts console through the vMA) see if you can get a response with the vmkping command.  With vmkping you should be able to ping your Group IP Address from the host if you can't it could be a VLAN or networking issue.

[FunkyD]

It turns out that the Dell instructions for using the Equallogic PS6000 with ESXi 5 are incorrect and VMware are in the process of discussing with Dell having them changed.

Bottom line is that you have to use MEM when using the Equallogic with ESXi 5. You cannot do one to one mapping between vmkernal ports and the network adapters as described in the Dell documentation.

[JESX35]

Dell still hasn't got this nailed down yet..??   Because when I last talked to them they said the MEM isn't "supported" for Vmware ESX5i unless you have enterprise plus, so unless they have changed their tone on this what do people do with PS series san that don't use Vmware Enterprise Plus ?

thought I got it working once through the vMA on ESX5i but it might of been 4i I can't remember now.

Anyhow, i'm glad you got this squared away.  Let us know what the final solution was

[AndreTheGiant]

Which kind of switches are you using? (there are some issue with the new 55xx series and vSphere 5).

IMHO try to use a single switch (for the test), check again jumbo frames (on vmkernel, vswitches and switches), do not use VLAN (if possible).

And I found also some issue with the round robin multipath, so I suggest to use the fixed one.

[FunkyD]

Well there is something going on here. I've had both Dell and Vmware look at this and both are scratching their heads.

I have two hosts and on the eql I've configured four volumes, chap authentication is set on two volumes with one username and on the other two volumes with another username.

One host sees the correct volumes under the adapter static discovery but not under storage, the other sees the wrong volumes under static discovery i.e. the ones it is not suppossed to be able to see.

Both hosts have the same chap user and password yet between them they see all four volumes!

What is puzzling is why the volumes do not show up under the storage adapter i.e. under devices.

I have been following the Dell document on MEM (the EPA release) - Installation and User Guide V1.1

There is also the Dell document http://www.equallogic.com/WorkArea/DownloadAsset.aspx?id=10799

This is the one that Vmware told me NOT to use as it will not work. According to Vmware, you have to use MEM because port binding does not support routing.

I have Vmware looking at this on Monday so hopefully we'll get some answers.

BTW, the switches I am using are Dell PowerConnect 6224 and I am using a VLAN ( I have to atm).

[dwilliam62]

This is not correct.  You do not have to use MEM to make iSCSI work with ESXi v5. A new TR for ESXi v5 is on the Equallogic website.  There's one for MEM v1.1 as well. Storage Modules like MEM still require an Enterprise or Enterprise Plus license.  That requirement comes from VMware.

There is one issue with ESXi v5 when you bind multiple NICs to the same subnet for iSCSI.  Routing no longer works.  It sounds like the VMKernel ports are on a different subnet from the EQL array.   The "no route to host" is a key indicator of that condition.  This was supportable in ESX(i) v4.x.

Check the subnet masks?

You must use one-to-one mapping of VMkernel ports to physical NICs in ESX v4.x -> ESXi v5.0.  If you don't then the GUI in v5 shows that as "unsupported" when you try to bind the VMK port to the iSCSI HBA.

What is the Dell case #?

-Don

[dwilliam62]

BTW:  I've confirmed that MEM does NOT resolve the ISCSI routing issue.  If you are routing to get to the arrays, it will not work with ESXi v5.0.

Are you VLAN tagging the iSCSI VMK ports?   If so, then make sure the ports the arrays are using are NOT tagged.  The switch will have to strip those tags off when connecting to the EQL arrays.

-don 

[FunkyD]

Hi,

There is no routing between the EQL storage and my iSCSI Vmkernel ports.

I am using the one-to-one mapping - this is done for me by the MEM utility.

Dell looked at it today and are still unsure exactly what is going on. Even a host without MEM isn't connecting to the storage and this was built from scratch.

What do you mean by the VLAN tagging? I am using a VLAN for iSCSI but there is no routing.

What I have done on the host that has MEM is increase MemberSessions to 4 and VolumeSessions to 12 and then the storage appeared. I don't know if it is coincidence or whether I have actually solved the problem yet so I'm going to try on another host.

Edit: A reboot later and now neither host can't see the storage again. This is bizarre! How can the host that I seemed to fix after a reboot no longer be able to see the storage? Nothing changed, all I did was reboot.

[cdickerson75]

My two cents.

So today I did a fresh install of vSphere 5 with the MEM 1.1 module and connecting to my Equallogic SANs.  I used the setup.pl script included with the MEM module to do all the configuration.  I am having no issues with my host seeing the SAN and all the volumes.   FYI, my Access security on the volume is set to CHAP and a IP subnet, but the IQN is *.  My issue is that vSphere is not using all 3 of the nic's in the vSwitch, it uses only 1.  Observed by looking at the NIC Performance Graphs. Under vSphere 4.1 and MEM 1.0, all 3 nics send traffic pretty evenly.  This has cut my perfomance in half.  A 18GB VM clone on vSphere 4.1 takes about 6 minutes, on vSphere 5 it took about 12 minutes.  On vSphere 5 I have one NIC doing about 22Kbps, under vSphere 4.1 I have three NICs doing a total of about 50Kbps.  I've opened a case with Equallogic because my gut is their MEM module isn't working yet.  My plan is to try this configuration with my PSP set to RR instead of EQL.

[dwilliam62]

With the default setting in MEM 1.x, when you have more than 2x NICs for iSCSI, not all NICs will be used to reach all volumes.  You will have to change the membersessions and volumesessions values to get all NICs utlitized.   This is covered in the MEM users guide.

If you do switch to VMware Round Robin, change the IOPs value to 3, the default is 1000.  It will only switch to another path until 1000 IOs have been processed.   So your nics will be less utilized with RR vs. MEM.

The 'naa.xxxxxxx' refers to the MPIO device name for the EQL volumes.

Setting default policy for EQL devices to Round Robin. So new volumes discovered will be set to Round Robin

#esxcli storage nmp satp set --default-psp=VMW_PSP_RR --satp=VMW_SATP_EQL

**These new volumes will still need to have the IOPs value changed.

To gather a list of devices use:

#esxcli storage nmp device list
You'll need the naa.<number> that corresponds to the EQL volumes in that list. That the "device number" that is used in the nextx command.

Existing volumes can be changed to Round Robin

#esxcli storage nmp device set -d naa.6090a098703e30ced7dcc413d201303e --psp=VMW_PSP_RR

You can set how many IOs are sent down one path before switching to the next. This is akin to rr_min_io under Linux.

NOTE: This will only work if the policy has been changed to Round Robin ahead of time.

The "naa.XXXXXXXXXXXXX" is the MPIO device name.

You can get a list of devices with:

#esxcli storage nmp device list

naa.6090a098703e5059e3e2e483c401f002
Device Display Name: EQLOGIC iSCSI Disk (naa.6090a098703e5059e3e2e483c401f002)
Storage Array Type: VMW_SATP_EQL
Storage Array Type Device Config: SATP VMW_SATP_EQL does not support device configuration.
Path Selection Policy: VMW_PSP_RR
Path Selection Policy Device Config: {policy=iops,iops=3,bytes=10485760,useANO=0;lastPathIndex=3: NumIOsPending=0,numBytesPending=0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba36:C0:T1:L0, vmhba36:C1:T1:L0, vmhba36:C2:T1:L0, vmhba36:C3:T1:L0

This also lets you confirm the path policy "VMW_PSP_RR" Which is VMware, Path Selection Policy, Round Robin" And not the IOPs value has already been set to '3'.

#esxcli storage nmp psp roundrobin deviceconfig set -d naa.6090a098703e30ced7dcc413d201303e -I 3 -t iops

#esxcli storage nmp psp roundrobin deviceconfig get -d naa.6090a098703e30ced7dcc413d201303e
Byte Limit: 10485760
Device: naa.6090a098703e30ced7dcc413d201303e
IOOperation Limit: 3
Limit Type: Iops
Use Active Unoptimized Paths: false

Regards,

-don

[cdickerson75]

Thanks for the info.  Not sure the default values are my problem.  I have 3 nics on the host, 3 nics on each member, and a two member group.  On my vSphere 5 host I see a total of 4 connections on each volume, which would make since because the maximum connections per slice is 2.  So with 2 connections I should at least see two of the three nics busy, I see only one.  Right? And the real gotcha, is under vSphere 4.1, it too has 4 connections per volume, but all three nics are used.  I have never had to tweak the MEM values under vSphere 4.1 or the MEM 1.0 version.

-Craig

[FunkyD]

This sounds like a different issue and probably worth starting another thread.

Thanks

I've now removed the VLAN tagging from vSphere and set membership on the switch to untagged. Scanning still shows the discovered volumes but no connection to storage.

[dwilliam62]

Hi Craig,

The settings don't sound like you're problem if you can't get at least two of the NICs used.  It's usually when you go to four NICs that you need to tweak MEM.

Have you opened a case with Dell?   Need to check the bindings and other settings.  If you set it to VMware Round Robin to get you connections from all three NICs?

Regards,

-don

[kpc]

Did anyone ever get a resolution to this?  We've just installed ESXi5.0.0 onto 3 new servers and experiencing the same things.  EQ volumes showing up in the static window but no details of them in the Details window.  Do a rescan they dissappear, a reboot brings one back - weird....

[dwilliam62]

Strongly suggest open a case with Dell/EQL.  Likely is related to a network or configuration issue.

Regards,

Don

[rtechnology]

Had the same issue, but got it resolved. Remove "inherit from parent", enter chap username and pass in chap settings (static discovery) for each LUN/Volume that you cannot map. Rescan and everything should work. Weirdest problem.

[uninspired]

rtechnology wrote:

Had the same issue, but got it resolved. Remove "inherit from parent", enter chap username and pass in chap settings (static discovery) for each LUN/Volume that you cannot map. Rescan and everything should work. Weirdest problem.

Remove "inherit from parent" from which setting? I'm having the same problem, but only with our Isilon storage. Equallogic, EMC VNXe both work without issue, but can't get the Isilon targets to work.

Thanks