Networking setup for vSphere

[ayking]

We are building a new vSphere ESXi setup and would like some input on the networking setup. Physical server would have 10 NICs. We are aiming for redundancy as the most important factor.

4 x NIC would go into vSwitch for iSCSI with multi-paths to two dedicated physical switches to our dual-controller SAN. Spanning Tree disabled on the switches, switches are connected but not stacked. NIC Teaming for this vSwitch is all NIC active with IP hash.

pNIC1 -> iSwitch1 -> SAN1

pNIC2 -> iSwitch2 -> SAN1

pNIC3 -> iSwitch1 -> SAN2

pNIC4 -> iSwitch2 -> SAN2

On a separate pair of physical switches we would have 3 VLANs/vSwitch:

2 x NIC would go into vSwitch for VMotion (pNIC5, pNIC6)

2 x NIC would go into vSwitch for management traffic (pNIC7, pNIC8)

2 x NIC would go into vSwitch for VM network traffic (pNIC9, pNIC10)

pNIC5, pNIC7 and pNIC9 would go into nSwitch1; pNIC6, pNIC8 and pNIC10 would go into nSwitch 2.

Both nSwitches are connected together but not stacked, and each nSwitch has an uplink to our core switch.

Does this sound right?

Should STP be enabled on the network switches? How should I set the NIC teaming/failover properties of the network vSwitch?

We are deciding between Dell PowerConnect 6224 and HP Procurve 2910al for the switches.

Thanks a lot!!

[Rajeev_S]

Welcome to the forums!

- STP should not be enabled

Below links should help you,

http://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf

http://communities.vmware.com/message/1277251

http://vmetc.com/2009/08/12/vswitch-with-multiple-vkernel-portgroups-for-vsphere-iscsi-round-robin-m...

http://blogs.netapp.com/virtualstorageguy/2009/07/vsphere-on-netapp-best-practices-released.html

http://yourmacguy.wordpress.com/2009/11/09/vsphere-iscsi-multipathing/

Award Points if helpful!!

[Texiwill]

Hello,

THis setup looks fine.

Please note that STP may need to be used, but that is why you enable Portfast (Cisco term) on all ports connected to an ESX host.

ESX vSwitch just does not understand STP so will ignore it but in a failure mode you just do not want the STP negotiations to take place so enable portfast on the ports in question. Or disable STP entirely... not usually an option.

Best regards,
Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, 2010

Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security'[/url]

Also available 'VMWare ESX Server in the Enterprise'[/url]

Blogging: The Virtualization Practice[/url]|Blue Gears[/url]|TechTarget[/url]|Network World[/url]

Podcast: Virtualization Security Round Table Podcast[/url]|Twitter: Texiwll[/url]

[ayking]

Thanks. I was quite sure on the iSCSI part of it as I've seen most of the links Rajeev posted. It's the regular VM network traffic that I'm unsure as most people talk about the iSCSI network but not many talk about the LAN setup.

[Texiwill]

Hello,

Most people state to separate the storage traffic from everything else using physically separate switches, etc. for security, performance reasons. Storage is its own trust zone.

Virtualization Management Network is its own trust zone. VMotion/FT are its own trust zone.

VM networks are at least 1 trust zone.

So at least 4 trust zones.

Your vNetwork configuration depends entirely on how many trust zones you have within it and how you segregate those trust zones within your physical network. If you use VLANs then use VLANs in the vNetwork. If you use physically separate switch fabrics then maintain that through your vNetwork using different vSwitches and pNICs.

The best suggestion is to NOT combine trust zones within the vNetwork unless you do so in the physical network.

However, you also need to consider performance and redundancy. For this it is generally recommended you have 2 pNICs per network or if you use 10Gb combine a bunch of networks onto those pair of pNICs....

Best regards,
Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, 2010

Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security'[/url]

Also available 'VMWare ESX Server in the Enterprise'[/url]

Blogging: The Virtualization Practice[/url]|Blue Gears[/url]|TechTarget[/url]|Network World[/url]

Podcast: Virtualization Security Round Table Podcast[/url]|Twitter: Texiwll[/url]

[kac2]

if you need some more configuration suggestions. I have some here on my blog vSphere Host NIC Design - 10 NICs