iSCSI MPIO with nexus 1000v and EqualLogic PS6000

[manfriday]

Hi,

Trying to set up an equallogic array with ESXi 4.1, with the nexus 1000v Version 4.0(4)SV1(3a).

I have experience setting up iscsi with a standard vSwitch, and a vmware dVSwitch, but my cisco knowledge is... limited to say the least.

We are trunking almost all of our traffic on a pair of 10Gb nics.

Here are my relevant port-profile configs:

port-profile Dist-Uplink

description:

type: ethernet

status: enabled

capability l3control: no

pinning control-vlan: -

pinning packet-vlan: -

system vlans: 878,880-881,3000

port-group: Dist-Uplink

max ports: -

inherit:

config attributes:

switchport mode trunk

switchport trunk native vlan 666

switchport trunk allowed vlan 32,70-71,104,107-108,110-112,851,878-881,3000

channel-group auto mode on mac-pinning

no shutdown

evaluated config attributes:

switchport mode trunk

switchport trunk native vlan 666

switchport trunk allowed vlan 32,70-71,104,107-108,110-112,851,878-881,3000

channel-group auto mode on mac-pinning

no shutdown

port-profile VM-iSCSI-3000

description: iscsi storage network

type: vethernet

status: enabled

capability l3control: no

pinning control-vlan: -

pinning packet-vlan: -

system vlans: 3000

port-group: VM-iSCSI-3000

max ports: 32

inherit:

config attributes:

switchport mode access

switchport access vlan 3000

capability iscsi-multipath

no shutdown

evaluated config attributes:

switchport mode access

switchport access vlan 3000

capability iscsi-multipath

no shutdown

I have two vmknics assigned to the VM-iSCSI-3000 port group. However, I only see one path.

When I run vemcmd show iscsi pinning I see both vmk3 and vmk4 going to different Pinned_Uplinks (18 and 20)

If I run esxcli swiscsi nic list -d vmhba34 (my software iscsi HBA) I see: Errors: No nics found for this adapter.

If I try running esxcli swisci nic add -n vmk3 -d vmhba34 (as I was used to doing for standard vswitch/iscsi setups) I get "Errors: Failed to add nic".

I CAN make a connection to the EqualLogic, but it is only a single path, and the performance is atrocious.. I did an IOMETER test of the equallogic lun and got back like 5-7Mb/sec with the 100% read test. which is pretty bad.

I am guessing I have something wrong with my nexus config but I am at a loss as to what the could be at this point. I'd appreciate any help you could offer.

[manfriday]

A bit of an update..

The "Dist-Uplink" port profile has two 10Gb nics assigned to it.

For grins, I removed one of the 10Gb nics and ran IOMETER again, this time getting 97MB/sec.

That is a little closer to what I would expect of a single path.

Adding the nic back to the uplink profile resulted in the poor performance again. It didnt matter which nic was in the profile. As long as there was only 1 the performance of the single path was about 97MB/sec. Adding the second one drove the performance down to 3-5MB/sec.

Any idea what this could mean?

[manfriday]

Ok, I think this is resolved. In case anyone ends up googling this, looking for an answer; this is what I discovered...

We had two issues.

The first issue is that when you try bind the iscsi VMK nics to the swiscsi HBA, you have to ensure that NO other vmk nics (vmotion, ft, etc.,) exist on your nexus 1000v.

Otherwise, when you issue the command: esxcli swiscsi nic add --adapter vmhba# --nic vmk#

You will get back "failed to add nic" or something along those lines..Removing all the other vmk nics allowed me to bind the iscsi vnk nics to the sw hba.My second issue was a misconfiguration on the upstream switches.Apparently we needed a port channel created which was not possible becuase we were plugged into two switches.one is a 6509 and the other is a n5k.So it looks like I can only build with redundancy back to the physical switches.until we pick up another n5k.