2 Replies Latest reply on Oct 15, 2010 2:14 PM by manfriday

    iSCSI MPIO with nexus 1000v and EqualLogic PS6000

    manfriday Hot Shot

       

      Hi,

       

       

      Trying to set up an equallogic array with ESXi 4.1, with the nexus 1000v Version 4.0(4)SV1(3a).

       

       

      I have experience setting up iscsi with a standard vSwitch, and a vmware dVSwitch, but my cisco knowledge is... limited to say the least.

       

       

      We are trunking almost all of our traffic on a pair of 10Gb nics.

       

       

      Here are my relevant port-profile configs:

       

       

      port-profile Dist-Uplink

        description:

        type: ethernet

        status: enabled

        capability l3control: no

        pinning control-vlan: -

        pinning packet-vlan: -

        system vlans: 878,880-881,3000

        port-group: Dist-Uplink

        max ports: -

        inherit:

        config attributes:

          switchport mode trunk

          switchport trunk native vlan 666

          switchport trunk allowed vlan 32,70-71,104,107-108,110-112,851,878-881,3000

          channel-group auto mode on mac-pinning

          no shutdown

        evaluated config attributes:

          switchport mode trunk

          switchport trunk native vlan 666

          switchport trunk allowed vlan 32,70-71,104,107-108,110-112,851,878-881,3000

          channel-group auto mode on mac-pinning

          no shutdown

       

       

       

       

       

       

       

       

      port-profile VM-iSCSI-3000

        description: iscsi storage network

        type: vethernet

        status: enabled

        capability l3control: no

        pinning control-vlan: -

        pinning packet-vlan: -

        system vlans: 3000

        port-group: VM-iSCSI-3000

        max ports: 32

        inherit:

        config attributes:

          switchport mode access

          switchport access vlan 3000

          capability iscsi-multipath

          no shutdown

        evaluated config attributes:

          switchport mode access

          switchport access vlan 3000

          capability iscsi-multipath

          no shutdown

       

       

       

       

       

      I have two vmknics assigned to the VM-iSCSI-3000 port group. However, I only see one path.

       

       

      When I run vemcmd show iscsi pinning I see both vmk3 and vmk4 going to different Pinned_Uplinks (18 and 20)

       

       

      If I run esxcli swiscsi nic list -d vmhba34 (my software iscsi HBA) I see: Errors: No nics found for this adapter.

       

       

      If I try running esxcli swisci nic add -n vmk3 -d vmhba34 (as I was used to doing for standard vswitch/iscsi setups) I get "Errors: Failed to add nic".

       

       

       

       

       

      I CAN make a connection to the EqualLogic, but it is only a single path, and the performance is atrocious.. I did an IOMETER test of the equallogic lun and got back like 5-7Mb/sec with the 100% read test. which is pretty bad.

       

       

      I am guessing I have something wrong with my nexus config but I am at a loss as to what the could be at this point. I'd appreciate any help you could offer.

       

       

        • 1. Re: iSCSI MPIO with nexus 1000v and EqualLogic PS6000
          manfriday Hot Shot

           

          A bit of an update..

           

           

          The "Dist-Uplink" port profile has two 10Gb nics assigned to it.

           

           

          For grins, I removed one of the 10Gb nics and ran IOMETER again, this time getting 97MB/sec.

           

           

          That is a little closer to what I would expect of a single path.

           

           

          Adding the nic back to the uplink profile resulted in the poor performance again. It didnt matter which nic was in the profile. As long as there was only 1 the performance of the single path was about 97MB/sec. Adding the second one drove the performance down to 3-5MB/sec.

           

           

          Any idea what this could mean?

           

           

          • 2. Re: iSCSI MPIO with nexus 1000v and EqualLogic PS6000
            manfriday Hot Shot

            Ok, I think this is resolved. In case anyone ends up googling this, looking for an answer; this is what I discovered...

             

            We had two issues.

             

            The first issue is that when you try bind the iscsi VMK nics to the swiscsi HBA, you have to ensure that NO other vmk nics (vmotion, ft, etc.,) exist on your nexus 1000v.

             

             

            Otherwise, when you issue the command: esxcli swiscsi nic add --adapter vmhba# --nic vmk#

             

             

            You will get back "failed to add nic" or something along those lines..Removing all the other vmk nics allowed me to bind the iscsi vnk nics to the sw hba.My second issue was a misconfiguration on the upstream switches.Apparently we needed a port channel created which was not possible becuase we were plugged into two switches.one is a 6509 and the other is a n5k.So it looks like I can only build with redundancy back to the physical switches.until we pick up another n5k.