"Best" Practice with Configuring iSCSI with VNX and Cisco UCS

[NullCollier]

I am looking for some assistance  with an implementation.  I am attemping to configure 4 vNICs in vSphere 5  for iSCSI to connect to 4 iSCSI ports defined on a VNX 5300 for a high  bandwidth configuration.

Attached  is a diagram, but here is what I am planning on doing and I would like  to know if anyone sees any issues with this implementation plan.

I  have a VNX 5300 with 2 10GB ports per SP.  I have defined 2 vLANs for  iSCSI traffic and have configured the SP ports as follows:

     SPA-0 with an address on vLAN 1

     SPA-1 with an address on vLAN 2

     SPB-0 with an address on vLAN 2

     SPB-1 with an address on vLAN 1

I  have 6 ESXi vSphere 5 hosts, each with 4 10GB vNICs dedicated for iSCSI  use.  On each ESXi 5 Host, I have defined 2 vSwitches for iSCSI, each  with 2 vNICs.  On vSwitch 1, vNIC 2 is configured for vLAN 1 and vNIC 5  is configured for vLAN 2.  On vSwitch 2, vNIC 4 is configured for vLAN 1  and vNIC 3 is configured for vLAN 2.

vNIC  2 of vSwitch 1 and vNIC 4 of vSwitch 2 (both on the vLAN 1 network) are  connected to Fabric Interconnect A (UCS) while vNIC 5 of vSwitch 1 and  vNIC 3 of vSwitch 2 (both on the vLAN 2 network) are connected to Fabric  Interconnect B (UCS).

On  the UCS Fabric Interconnect A, I have configured 2 Applicance ports and  connected VNX SPA-0 and VNX SPB-1 (both on the vLAN 1 network).  On the  UCS Fabric Interconnect B,  I have configured 2 Applicance ports and  connected VNX SPA-1 and VNX SPB-0 (both on the vLAN 2 network).

Given  this configuration, would all 4 ports of the VNX and ESX Hosts be  utilized for iSCSI I/O assuming that the ESX host would connect to  multiple LUNs on the VNX which would be spread evenly between SPs?

What  I do not want to happen is have iSCSI traffic from Fabric Interconnect A  attempt to connect to SP ports only connected to Fabric Interconnect B  by attempting to travel upstream and come back down to the other Fabric  Interconnect.  Given what networking gear is upstream this would cause a  serious performance issue.

Anyone see any issues with this design?  Thoughts and/or comments?

[Claguey]

Hi,

How did you go with the configuration you proposed? Would be realy interested how you went on.

We have the exact same setup of a VMWare, UCS and VNX 5300 with iSCSI for both Boot and storage. We are having a really problem where logins to the VNX from the UCS blades are randomly logging out, this then hangs off for around 15 mins and then logs back in.

Are you going through a network switch from the VNX to the Fabric interconnects?

Regards

Andrew

[NullCollier]

Andrew - After much discussion we ended up using only 2 NICs for iSCSI traffic from the vSphere Host.  Attached is a diagram that outlines how we implemented the solution. We talked to VMware and they informed me that only up to 2 NICs for Software iSCSI was supported.  This might be mentioned some where but I must have missed that.

We have not experienced the problems you are talking about.  Our solution seems to be working just fine.

I have set up UCS/VMware/iSCSI solutions that use an upstream network swtch for both LAN and iSCSI traffic and I have implemented these solutions where the iSCSI storage is directly connected to the UCS Fabric Interconnects using the Appliance Port features.  Both ways seem to work just fine.  However if I have the flexibility, I prefer to direct connect the storage to eliminate any extra hops and/or network complexities.

Hope this helps.

[Claguey]

Thanks for the reply!

I think out of courtesy, i should reply with what we ended in doing.

We had a setup that was very similar to the above configuration. We were using Cisco 4507e chassis switches to pass the iSCSI and LAN traffic between the VNX 5300 and the UCS. After packet captures and logs from both the VNX, UCS and ESXi , we decided to do a full reload of the 4507E. This then stopped the logouts completely in one site and vastly reduced them in the other.

Taking that this was very proplematic, we decided away from iSCSI and went to fibre channel. We since have not had a problem.

Regards

Andrew

[NullCollier]

Andrew - Glad you got your problem resolved.  I have not introduced a Cisco 4000 series switch into the mix but have used a Nexus 5k to connect a VNX to a Cisco UCS environment.

What I have found is that connecting a VNX directly to the UCS FI's works fine for iSCSI (using appliance ports) and connecting a VNX to a Nexus 5k for iSCSI works fine with UCS.

Glad your problem has been resolved though.