vSphere network design - thoughts?

[lcfc2]

Hi All,

Looking for some advice/confirmation on a build I'm puting together

Hardware Blade C3000, 4 Passhru switches

x2 BL460c G6 servers with 6 NICS each.

Design is looking like the following,

2 teamed NIC's for production, service console on same vSwitch0, thoughts on this?

1 NIC on each host for Vmotion

x2 NIC ISCSI SAN and SC (not used ISCSI before just FC)

1 NIC DMZ on each host

Would this work ok? Using PassThru to keep things simple and patch to core switch and the DMZ side of things. Any security concerns?

Look forward to some input.

Cheers

[vmroyale]

Hello and welcome to the forums.

Check out "[Blue Gears - DMZ w/6 Physical NICs with VMware ESX|http://www.networkworld.com/community/node/39307]" by Edward Haletky. It addresses many of your concerns.

Good Luck!

[lcfc2]

Thanks vmroyale, interesting article.

I think the first solution outlined is the closest to what I require.

pNIC0 -> vSwitch0 -> Portgroup0 (service console)

pNIC1 -> vSwitch0 -> Portgroup1 (VMotion)

pNIC2 -> vSwitch0 -> Portgroup2 (Storage Network)

pNIC3 -> vSwitch0 -> Portgroup3 (VM Network)

pNIC4 -> vSwitch1 -> Portgroup4 (DMZ Network)

pNIC5 -> vSwitch1 -> Portgroup4 (DMZ Network)

Am I missing something here though, redundancy on the VM network (production) & Storage network?

My outline would be like this:

pNIC0 -> vSwitch0 -> Portgroup0 (SC/Production)

pNIC1 -> vSwitch0 -> Portgroup0 (SC/Production)

pNIC2 -> vSwitch0 -> Portgroup1 (VMotion)

pNIC3 -> vSwitch1 -> Portgroup2 (Storage network, ISCSI/SC)

pNIC4 -> vSwitch2 -> Portgroup3 (Storage network, ISCSI/SC)

pNIC5 -> vSwitch3 -> Portgroup4 (DMZ)

I'm not too concerned about providing redundancy on the DMZ, I guess my concern is having the SC on the production vSwitch.

Thanks

[chadwickking]

Well Lcfc,

we use a 6 nic config but with "no" DMZ. I am sure we will when we get there but we will provision more nics. You can never have too many Nics for your esx host and actually its one of the top 3 problems users run into in future upgrades/deployments. Having dmz takes away from your nic redundancy but its something you obviously will need inyour set up.

Cheers,

Chad King

VCP-410 | Server+

Twitter:

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

[habibalby]

Hello,

Since you are hosting a DMZ in your infrastucture, remember securing your COS is highly recommended. have a look at these threads, might help you to decide what to do.

ESX Networking, S.C, VMotion, Production DMZ???

Firewall Between ESX-vCenter vLAN & Production vLAN

Best Regards,

Hussain Al Sayed

Revisit your posts and award points for "correct" or "helpful".

[lcfc2]

Thanks habibalby,

So what i'm reading tells me I need more Nic's? Don't think that is going to happen so a compromise in the deisgn somewhere instead. The security team may insist on the SC not being on the production LAN, remember I'm using Pass Thru's.

Have my VC behind a firewall? My VC will be a virtual machine on one of the ESX boxes.

[AndreTheGiant · Accepted solution]

Use 2 NIC only for iSCSI... not SC.

So you have 4 NIC free and you can consider (if possible) to use VLAN tagging to have vMotion, DMZ, Management, LAN on the same vSwitch.

Then for each port group use the right VLAN tag and bind it on different NIC.

Andre

[Texiwill]

Hello,

You can do this all with 6 pNICs but you will need to make some hard choices.

So what i'm reading tells me I need more Nic's? Don't think that is going to happen so a compromise in the deisgn somewhere instead. The security team may insist on the SC not being on the production LAN, remember I'm using Pass Thru's.

The SC SHOULD NEVER be on the production LAN directly....

Have my VC behind a firewall? My VC will be a virtual machine on one of the ESX boxes.

Good, now place each SC behind the same firewall. This is the most important thing you can do to increase security. Just firewalling vCenter does NOT actually protect you from much... .why? Because vCenter communicates with each ESX host and each vSphere CLient communicates with vCenter. So you have protected (not very well) a single point of the 3 major points to protect. Which means traffic outside that firewalled vCenter can still be attacked. I know some pentesters who could own your systems with this configuration in less than 2 minutes.... Not a good picture.

So instead of thinking of pNICs, think of Trust Zones..... You have these trust zones:

Virtualization Management (SC, vCenter, vSphere Client, ESXi Management Appliance)

Storage (iSCSI, NFS, FC)

DMZ

Production Network

vMotion/Fault Tolerance

So what is acceptable to combine? I would say combine your Virtualization Networks onto one set of pNICs, DMZ and Product on others.

Virtualization Networks are the Virtualization Management, Storage, and vMotion/Fault Tolerance.

However, this will greatly impact Storage if you are using iSCSI or NFS as you ideally want 2 pNICs just for Storage.

Or you could combine DMZ/Production on the same set of pNICs using VLANs, but this is ONLY acceptable if you are already using VLANs for this type of segregation in your existing physical network. If you are not, then you need to assign at least a pNIC to this.

The key is that the vNetwork is an extension of your physical network and what you do within your physical network defines what you can do within your virtual network. If you segregate with VLANs in the pNetwork, continue this in the vNetwork.

Best regards,
Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, 2010

Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security'[/url]

Also available 'VMWare ESX Server in the Enterprise'[/url]

Blogging: The Virtualization Practice[/url]|Blue Gears[/url]|TechTarget[/url]|Network World[/url]

Podcast: Virtualization Security Round Table Podcast[/url]|Twitter: Texiwll[/url]

[Dr_Virt]

This is where Xsigo really pays dividends...

But I digress, I would join others by recommending the consolidation of trusted zones. Three vswitches with 2 interfaces each - VMware (SC, vMotion, etc), Traffic (DMZ/Production VLANs), and Storage (iSCSI/NFS).

If you do have some budget, I would recommend looking into Xsigo. With this you have the option of up to 128 network interfaces per blade over 10G NICs or Infiniband. We have solved a number of complex issues utilizing this solution.

[lcfc2]

Thanks all for the reply, very helpful (who gets the points).

I would like to use VLANS but people in the team have informed me they couldn't get it to bind properly previously using HP Blades and chasis, I've seen it work with HP so any ideas as what might cause the VLAN bindig to fail? Port trunking needs to be enabled on the switches. What else? PortFastmode? Extra info - all Trunk ports are shown as VLAN 1 on Cisco Catalyst 6000 switches.

With regard to firewalling the SC would this in effect be creating a firewall between the management lan and production lan?

By the way Ed I'm just waiting for your book 'VMware vSphere(TM) and Virtual Infrastructure Security' to arrive, should be of some assistance.

Cheers

[Texiwill]

Hello,

I would like to use VLANS but people in the team have informed me they couldn't get it to bind properly previously using HP Blades and chasis, I've seen it work with HP so any ideas as what might cause the VLAN bindig to fail? Port trunking needs to be enabled on the switches. What else? PortFastmode? Extra info - all Trunk ports are shown as VLAN 1 on Cisco Catalyst 6000 switches.

ESX just requires the following:

Trunk through the pSwitch to the vSwitch (if using VST which would be recommended)

Enable Portfast on the edge pSwitch which implies the one in the blade chassis usually but if you are using virtual connect you may need to do it on the Cisco switch... Not possitive there.

With regard to firewalling the SC would this in effect be creating a firewall between the management lan and production lan?

Absolutely the best way to go. The Virtualization Management Lan would be firewalled from the production LAN and within this firewall you would place vCenter, vSphere Clients, vSphere SDK tools, etc.... The Virtualization Administrator would RDP to a jump machine where these tools reside and resides within this firewalled area. That way they do not need to have any such tools on their desktop. They are most likely use to using RDP so this does not affect usability. These jump machines can also be VMs btw. The added benefit is that this works if you want to VPN in to fix things as well.

By the way Ed I'm just waiting for your book 'VMware vSphere(TM) and Virtual Infrastructure Security' to arrive, should be of some assistance.

I hope it will be. Let me know if you have any questions about the book contents.

Best regards,

Best regards,
Edward L. Haletky VMware Communities User Moderator, VMware vExpert 2009, 2010

Now Available: 'VMware vSphere(TM) and Virtual Infrastructure Security'[/url]

Also available 'VMWare ESX Server in the Enterprise'[/url]

Blogging: The Virtualization Practice[/url]|Blue Gears[/url]|TechTarget[/url]|Network World[/url]

Podcast: Virtualization Security Round Table Podcast[/url]|Twitter: Texiwll[/url]