Networking configuration recommendations for 4 pNICs

[oturn]

Our ESX servers all have a total of 4 pNICs, and they are assigned to single vSwitch. All of the pNICs are active. Our VMs are on 3 separate VLANs (depending on which department in the organization is using the VM), and the SC and VMotion are both on a single management VLAN.

If best practice is to separate VMotion and SC traffic onto a separate networks (separate from themselves), with VMotion traffic being on a non-routable network, how should we go about doing this, considering we only have 4 pNICs?

Would it be acceptable to simply change the current VMotion/VMKernel port on the existing vSwitch to a new non-routable VLAN, or should we create a single new vSwitch using one of the 4 pNICs, assigning VMotion to this vSwitch (on a new, non-routable VLAN)? In other words, can we use VLANs on a single vSwitch to accomplish this, or do we need to split out a NIC or two into another vSwitch?

Comments please!!

[ncarde · Accepted solution]

vSwitch0: vmnic0 and vmnic1 (maps to phy nic0 and nic1)

Create a port group for Service console and override load balance to bind to vmnic0

Create a port group for VMotion and override load balacne to bind to vmnic1

vSwitch1: vmnic2 and vmnic3 (maps to phy nic2 and nic3)

Create a port group for Virtual Machine subnet

Summary:

(2) Separate Virtual Switches with failover redundancy

(3) Separate logical VLANs (1 for Console, 1 for VMotion and 1 for Virtual Machines)

[oturn]

Sounds reasonable. Does anyone else care to chime in?

[Texiwill]

Hello,

It does sound reasonable. It is the way I would do things as well. However, you will have to use VLANs if you want to share SC/vMotion on the same vSwitch.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[TomHowarth]

vSwitch0 = vmnic0 + 1

service console in one portgroup and vlan tagged

VMkernal/vMotion in second portgroup and vlan tagged.

have a read of this Scott Lowe blog on how to protect vmotion traffic

vSwitch1 = vmnic 2 = 3

VM guest traffic. create portgroups for seperate vlans. have a read of Scott Lowe blog for infomation on how to do configure VLAN tagging and Trunking

this is about the best you can do with 4 nics

Tom Howarth

VMware Communities User Moderator

[oturn]

Thank you all for the great advice.

I still need to clearly understand the benefits of this change. Won't this reduce the net bandwidth available to my VMs by half? Considering this reduction, how is this solution more beneficial than keeping all 4 pNICs assigned to 1 vSwitch, and separating all VC and VMotion traffic using VLANs? Is it all about keeping the occasional blast of VMotion traffic away from my VMs? If so, should I weigh making the change based on the amount of VMotion(ing) occurring, versus the halving the bandwidth to my VMs, or does best practice say I need this change regardless?

[Texiwill]

Hello,

You do not need to make any change but it is the best practice. Remember vMotion should be as fast as possible... Yes this is the best practice.

Your used bandwidth is something much less than what you are using. In general isolation of networks is suggested for security, performance, and redundancy sake.... Having all 4 pNICs using the same vSwitch can be done, but from a security perspective it is frowned upon.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization