VMware Cloud Community
ddorbuck
Contributor
Contributor

Question on iscsi networking. One vSwitch with multiple nics for iSCSI or seperate vSwitch's with individual nics?

Hello, I have been toying with the idea on setting up my new vsphere 5 servers iSCSI interfaces using one vSwitch and multiple Vmkernal port groups inside it for each iSCSI adapter or keep each one separate in their own vSwitch and vmkernal port group. I have it configured as the second way with each iSCSI adapter is in its own vSwitch on my existing vsphere 4 servers and it works fine with round robin and multiplexing.

The vmware 5 documentation lists each as an acceptable implementation. Is there any speed or performance difference from one configuration over the other? I have 4 gig nics allocated to iSCSI traffic. I am using the software iSCSI adapter since I will be using jumbo frame support.

http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc_50%2FGUID-0D3112...

Thank you. Doug

Reply
0 Kudos
8 Replies
DavidKlee
Enthusiast
Enthusiast

Because of the way I like to set up iSCSI networking on vSphere 5 and bind to the VMKernel ports with jumbo frames, I like to set up my iSCSI ports all on the same vSwitch. On vSphere 4, I liked to go the other method.

David Klee | Founder and Chief Architect | Heraflux Technologies | dklee@heraflux.com
ddorbuck
Contributor
Contributor

Thanks David. I see it appears more of a management or consolidation thing now in vsphere 5. Having a server with 12 gigabit ports or more the networking tab can get plenty busy with separate vSwitches for each.

Thank you. Doug

Reply
0 Kudos
DavidKlee
Enthusiast
Enthusiast

Wow - 12 ports! That's great Smiley Happy That would definitely get busy, especially if a lot of VLANs were involved.

David Klee | Founder and Chief Architect | Heraflux Technologies | dklee@heraflux.com
Reply
0 Kudos
rickardnobel
Champion
Champion

ddorbuck wrote:

using one vSwitch and multiple Vmkernal port groups inside it for each iSCSI adapter or keep each one separate in their own vSwitch and vmkernal port group.

Actually there has been some bugs in ESXi 5 with the iSCSI multipath binded to the same vSwitch and even if setup correctly with active/non-used there has been strange issues. It might be resolved in Update 1, but I am not really sure - and for this reason I recommend you to use multiple vSwitches with a single VMK interface connected.

My VMware blog: www.rickardnobel.se
DavidKlee
Enthusiast
Enthusiast

Interesting. Can you point me in the direction of the KB articles outlining the bugs? I have not run across any in this configuration, but am very interested to learn more details in case I do run across them in the field.

David Klee | Founder and Chief Architect | Heraflux Technologies | dklee@heraflux.com
Reply
0 Kudos
ddorbuck
Contributor
Contributor

Thank you sir. Since Im still waiting for my other team to configure the cisco switch for me this is a great time to pull the vSwitches out and reconfigure them.  Doug

Reply
0 Kudos
rickardnobel
Champion
Champion

David Klee wrote:

Interesting. Can you point me in the direction of the KB articles outlining the bugs? I have not run across any in this configuration, but am very interested to learn more details in case I do run across them in the field.

It all seems to work fine as long as all paths are up, but if a network cable was pulled and then restored the first path would not go back to the expected vmnic (with Fixed policy) and when looking in ESXTOP a vmknic could now used a vmnic which was set to "unused". Really strange.

See this KB: http://kb.vmware.com/kb/2008144

My VMware blog: www.rickardnobel.se
Reply
0 Kudos
DavidKlee
Enthusiast
Enthusiast

Interesting. That explains why I have not seen it in production. I always set roun robin pathing policies instead of fixed. Thanks for the info! I appreciate it.

David Klee | Founder and Chief Architect | Heraflux Technologies | dklee@heraflux.com
Reply
0 Kudos