10gbe Switches and Design Considerations

[admin]

I am interested in how others are currently deploying or are planning to deploy 10gb switches in their environments. In our environment we have two 48port 3750e's that our equallogic SAN's (seven members) connect to. These switches are cascaded to two additional 3750e's in our esx rack that our esx hosts connect to for SAN access. We also have two 3750e's that supply network access for our hosts.

I have seen a lot of discussion of people stating that 2 or (possibly 4) 10gb connections per host would be sufficient to handle all traffic using vlans. I am wondering if others feel that 2 x 10gbe switches with 48 ports would have the sufficient buffer and backplane capacity to handle the load of both vm network AND iscsi/SAN traffic combined on the same switch. So in essence doing away with the current 6 x 3750e switches and going with only 2 x 10gbe switches? My consideration is the cost of the both outfitting existing hosts with 10gbe adapters, as well as the considerably more expensive price tag on the switches.

Interested to hear how others are approaching this and how far off you feel you are from making the change to 10gig.

[RBurns-WIS]

Have a look through this whitepaper.  It speaks in reference to the 1000v, but you can apply the design conderations to vSS or vDS.

Regards,

Robert

[alevinson]

We are getting ready to build our next environment on all 10G.  We are using DL580 G7, EqualLogic 6510, and Nexus 5596.  Right now we are having difficulty in selecting a 10G NIC for iSCSI.  HP's cards have odd chipsets, not sure which is the best route, neither is HP; Mellanox, Qlogic, NETXEN and ServerEngine.  EqualLogic is suggesting iNTEL, but none of the HP OEM NICs have this chipset.  We will be converting 2 onboard NICs with the dual-port NC524 to 10G for VLANed LAN traffic, and the dual-port PCIe card will be strictly iSCSI.  This is not a cheap build, performance is key over price. 

[TheHevy]

I need to start off by stating that I work for Intel in LAN Access Division on 10Gb Ethernet Controllers and Virtualization technologies.

With that said, if you are interested in performance I would suggest taking a look at the new Tech Paper that VMware just published on Network Performance in vSphere 4.1. In the test, VMware uses the Intel Ethernet Server Adapter X520 which is based on 82599 10Gb controllers...

http://www.vmware.com/files/pdf/techpaper/Performance-Networking-vSphere4-1-WP.pdf 

Additionally, I would take a look at the SPECvirt results where Intel Ethernet Controllers are used by a lot of the OEMs...

http://www.spec.org/virt_sc2010/results/specvirt_sc2010_perf.html

I have three different white papers discussing 10Gb Ethernet in a virtual environment. Here is a summary and the links on a Blog post that Patrick Kutch did...

http://communities.intel.com/community/wired/blog/2010/12/08/latest-intel-ethernet-virtualization-pa...

I hope this helps...

Brian Johnson

Product Marketing Engineer, 10GbE Silicon - LAN Access Division

Intel Corporation

Twitter -- http://twitter.com/thehevy