Better use of multiple NIC's (work together with VLAN's or physcially separated)

[mikeyes]

Setting up my new production vSphere envrionment and I'm trying to figure out the best way to setup the network. I have pictures to illustrate but the basic quesion is:

1. Use all NIC's in a pool and use VLANs to separate the traffic -or-

2. Dedicate some physical nic's to only certain things (VMotion, FT etc.)

We are using 2 Dell R710 servers with 6 NIC's each.

***Our SAN's are connected via FiberChannel so the iSCSI you see in the group is only for emergency failover if the FC environment were to quit for any reason.

Please let me know which design you think would be the best.

Thanks,

Michael

[tonyholland_00]

Option 2 is a great way of doing it. You for sure want a dedicated nic for FT and VM. I would make sure that your primary Port Group has nics from 2 physicaly different nics in case of one failing.

Other than that you look good.

[mikeyes]

How about a third option?

Should the FT and VMotion NICs be dedicated or can I group them to take advantage of some physical switching redundancy?

(pic included)

[dsseagren]

I think option 3 looks the best. You will want to have some physical nic redundancy for critical functions like vmotion.

[mikeyes]

If I have some machines using FT and then try a VMotion will VMware handle the traffic well or will one process kill the other?

Thanks

[TomHowarth]

You have six nics in the machine, firstly you should separate your Service console traffic form your VM traffic.

vmnic 0 + vmnic2 = Service console and vmotion and FT (three portgroups

Vminc 1 + vmnic3 = VM Network

vmnic 4 and vmnic 5 = iSCSI traffic.

this give resilliance across networks, of you are using an intel 1000T make sure that the iSCSI traffic is at the least split across the dual onboard controllers.

If you found this or any other answer useful please consider the use of the Helpful or correct buttons to award points

Tom Howarth VCP / vExpert

VMware Communities User Moderator

Blog: www.planetvm.net

Contributing author on "[VMware vSphere and Virtual Infrastructure Security: Securing ESX and the Virtual Environment|http://www.amazon.co.uk/VMware-VSphere-Virtual-Infrastructure-Security/dp/0137158009/ref=sr_1_1?ie=UTF8&s=books&qid=1256146240&sr=1-1]”.

Contributing author on "[VCP VMware Certified Professional on VSphere 4 Study Guide: Exam VCP-410|http://www.amazon.co.uk/VMware-Certified-Professional-VSphere-Study/dp/0470569611]”.

[mikeyes]

Everyone says to separate the service console from the VMnetwork. If I can separate the service console traffic with a vlan why separate it physically? If something happens that the service console link gets disconnected but my machines are still communicating then I would have trouble. Why not make sure that if the network paths are up for the machines that I can control the VM server?

From the original post I have combined the iSCSI traffic because it is an emergency failover only in case my FiberChannel hardware has a problem. The iSCSI link will seldom to never get used and I didn't want to dedicate 2 physical NICs to something that would almost never be used.

Let me know what you think on the service console.

[Texiwill · Accepted solution]

Hello,

Everyone says to separate the service console from the VMnetwork. If I can separate the service console traffic with a vlan why separate it physically? If something happens that the service console link gets disconnected but my machines are still communicating then I would have trouble. Why not make sure that if the network paths are up for the machines that I can control the VM server?

Your VMNetwork is one of the most hostile networking environments within the vNetwork as its arbitrary and an attack point if some one breaks into a VM. If you have the Virtualization Management Network attached to the one your VMNetwork is on, then there is a VERY good chance that the hacked VM will now be used to launch an attack against the virtualization management network. Given the current set of attacks there is a VERY good chance of this succeeding. For Security reasons you want your Virtualization Management Network to be segregated and firewalled from any other physical and virtual network. Ideally on its own switches with out physical switch VLANs in use. However if you do use physical switch VLANs then you are placing your trust in these switch configurations, so you want to increase your monitoring of these switches.

From the original post I have combined the iSCSI traffic because it is an emergency failover only in case my FiberChannel hardware has a problem. The iSCSI link will seldom to never get used and I didn't want to dedicate 2 physical NICs to something that would almost never be used.

You want to dedicate 2 links to iSCSI, if you ever do have a failover you will want the bandwidth and redundancy. Redundancy should be considered for all storage links.

Let me know what you think on the service console.

When you use VLANs in the vNetwork you are automatically protected from most known Layer-2 attacks, but within the pNetwork you are trusting that your switch configurations will protect you. These configurations have been known to change and not necessarily for the better. Some say, you have to break so much to make this happen, but 1 misconfiguration and your SC is now under attack. Remember, once the virtualization management networks can be attacked they most likely can be broken. I know one pen-tester that can do this in very little time and they will own your virtual environment.

Protect the service console/management appliance, vSphere Client, vCenter servers as if they were gold, access to them implies access to nearly everything. This is why VMware strongly recommends that you create a separate Virtualization Management Network firewalled from all other networks on your system. That within this firewall you place jump machines which run all vSphere SDK and vSphere clients and that you use something like RDP to access these tools without running them through your firewall. Doing this one thing increases your overall virtual environment security by leaps and bounds protecting your investment from the current batch of management network attacks. VLANs are not a security tool, they are a network separation tool that relies on the pNetwork being properly configured, maintained, and audited. VLAN Security is based on trust in your pSwitches not something authoritative.

Best regards,
Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, 2010

Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security'[/url]

Also available 'VMWare ESX Server in the Enterprise'[/url]

Blogging: The Virtualization Practice[/url]|Blue Gears[/url]|TechTarget[/url]|Network World[/url]

Podcast: Virtualization Security Round Table Podcast[/url]|Twitter: Texiwll[/url]