ESX Networking, S.C, VMotion, Production DMZ???

[habibalby]

Dear All,

I'm on the process of building a Vi3 Infrastructure that consists of 2 BL-460 C Class. Each with 20 gig of ram, 6 pNICs and Qlogic FC 26242 HBA dual port via the mezzenien card.

My Question is:

The networks that I'm planning to put as follows:

1. Service Console

2. Production VMs

3. DMZ VMs

4. VMotion

Since I'm having 6 NIcs, how I shall be configuring those nics with regards of their service(s)?

Option 1:

1 For Service Console?

1 For vMotion ?

2 For Production VMs Network?

2 For DMZ VMs Network?

What about Failover in the Service Console and VMotion?

Or with VLANs, and 6 ports for each server!!!

Option 2:

1 for SC/vMotion Vlan?

2 for Production Vlan?

2 for DMZ Vlan?

If I combined the SC along with the VMotion Network, then the VMotion Network will be as part of the Service Console "where it's setting the Production Network"

Assume that, my production network is 128.104.0.0 then the Service Console has to be on the Same Subnet as well as the VMotion Network and the Production VMs Network??

Or I shall make the First Option without Failover for the S.C and Vmotion and assign the S.C Network a Production IP but for the VMotion Different IP? How the Hosts will talk to each other the S.C on Different IP and Vmotion Network on Different IP?

Could some please clarify this ?

[oreeh]

See

[Texiwill · Accepted solution]

Hello Habibalby,

The networks that I'm planning to put as follows:

1. Service Console

2. Production VMs

3. DMZ VMs

4. VMotion

Ideally you need 8 pNICs but you have only 6. I would do the following:

1 pNIC for SC for vSwitch0 using a VLAN

1 pNIC for vMotion for vSwitch0 using a VLAN

2 pNICs for production for vSwitch1

2 pNICS for DMZ for vSwitch2

The SC and vMotion back each other up when there is a failure in one of the pNICs hence the need for VLANs. you should not have the SC able to see vMotion traffic in a highly secure environment.

I would put the SC on an administrative network separate from production. I would also place vMotion on its own private VLAN not accessible by anything. This will give you the best protection.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[habibalby]

Hello Edward,

Thanks for your answer. My concern about the IP Addresses assignment for the Networks and thier routing with other subnet.

Putting the SC Network on VLAN in the pSwitch and give it another IP Address than the currently used "Production" and communicated with the VC?

So the VC is part of the Production Network, if I put the SC on seprate IP Schema than the production, I should only allow the VC accessing the SC and Administrators for Administrative Tasks?

What if, I give the SC the same IP As the Production Network? Is there any immplication or only Maintaining a Security by not allowing all the Production VLAN accessing the SC?

you should not have the SC able to see vMotion traffic in a highly secure environment.[/quote]

Also here I have to give another IP Address Schema? So, only the Hosts on this VLAN will be able to communicate with each other by making a Routing between the Production VLAN and SC VLAN?

Current Production IP: 128.104.0.0/16

DMZ in a Back-to-Back between Pix and ISA setting in between the DMZ and Production: 192.168.1.0/24

Thank you,

Habibalby

[Texiwill]

Hello,

Putting the SC Network on VLAN in the pSwitch and give it another IP Address than the currently used "Production" and communicated with the VC?

Yes, ideally VC would be on the same side of the firewall that the SC is and you would use a firewall to access VC and proxy VIC requests to the systems through VC.

So the VC is part of the Production Network, if I put the SC on seprate IP Schema than the production, I should only allow the VC accessing the SC and Administrators for Administrative Tasks?

You would need to use some form of gateway which could be a virtual appliance or a physical firewall. I would move VC to the same network as the SC.

What if, I give the SC the same IP As the Production Network? Is there any immplication or only Maintaining a Security by not allowing all the Production VLAN accessing the SC?

You can. I tend to want to secure the SC from the production network onto its own administrative network. It can be any IP scheme you want to use on the Administrative network with a firewall between it and the production network. I would do this, because production networks are often attacked as well generally from inside the organization. But whether you go this route depends on your security policy.

you should not have the SC able to see vMotion traffic in a highly secure environment.[/quote]

Also here I have to give another IP Address Schema? So, only the Hosts on this VLAN will be able to communicate with each other by making a Routing between the Production VLAN and SC VLAN?

No. vMotion has its own network that is not part of any other network. Granted it can be routed, but I like non-routing IP for vMotion. the only systems on the vMotion network are the ESX servers.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[habibalby]

Hellp Edward,

I have drawn my understanding of the pNICs assignment to it's corrosnbondace network for the SC & VMotion, Production and DMZ Network. Please have alook on the attched diagram.

Rack Diagram

Thank you very much for your support.

[Texiwill]

Hello,

I would have your vmnic1 also on pSwitch1, remember it will provide redundancy to the SC if pSwitch0 dies for some reason. As long as the VLANs are setup properly then all should be well.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[habibalby]

Yes, very good thank you I will correct my diagram to be the vmnic1 > to pSwitch0.

I really appreciate your help Edward.

BR,

Habibalby

[habibalby]

Hello once again,

I have another concern since i will be using the 6 pNICs in ESX 3.5. The new release 3.5 is require a Redundant SC. How I will be configure second SC if all my pNICs are occupied?

[Texiwill]

Hello,

You would place the SC on one VLAN and use pNIC0 and vMotion on another VLAN using pNIC1. pNIC0 would be the redundant for pNIC1 and pNIC1 would be the redundant for pNIC0. All this on the same vSwitch. But you must use VLANs to make this work.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[habibalby]

Hello Edward,

Thank you for your reply.

in my scenario i will be having vSwitch0 contains "SC (vmnic0) and VMkernal (vmnic1) via VLAN on different Portgroup and Different IP Schema.

i;e, if I have vSwitch0 contains SC portgroup on 128.104.30.0 segment "vmnic0″ and 10.0.0.2 for VMkernel portgroup on "vmnic1″ i won't be able to add another SC to the same vSwitch0, yeah!!!

Or i should make saperate vSwitch for each Network "SC and VMkernal"?

In this case, the result will be, vSwitch0 contains SC portgroup on 128.104.30.0 "vmnic0″ and vSwitch1 contains VMkernal portgroup on 10.0.0.0 "vmnic1″

If i add an addition SC to the vSwitch1 which it on Network 10.0.0.0 and my Original SC on 128.104.30.0 Network, what IP Address i should give the second SC on vSwitch1? is it 10.0.0.x IP and this IP should reach to the 128.104.30.0 Network where my production is setting as well as the VC?

Or, since the SC will be the same IP schema as the Production IP Schema I should make;

vSwitch0 contains only SC on "vmnic0″ on 128.104.30.0 AND

vSwitch2 contains Production VMs portgroup on "vmnic2, "vmnic3″ on 128.104.30.0 Network as well as Adding another SC portrgoup on 128.104.30.0 to the vmnic2 and vmnic3 on the same vSwitch2?? will that be possible? I know it's not a best practice security implementation, but that will work yeah?

Thanks for your suggestion

Best Regards,

Hussain Al Sayed

[Texiwill]

Hello,

I would not make the SC the same as the production network, as you will make multiple possible attack points into your SC. Not a suggested mechanism. Your SC should be separated from the production network via firewall.

SC/vMotion can share the same vSwitch, but I would not place the SC on any other vSwitch.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization

[habibalby]

Hello,

So, to summrize this.

vSwitch0 contains SC and VMotion each on it's own Portgroup and vmnic.

vmnic0 = Service Console in the IT Management VLAN that's spreated from Production Network

vmnic1 = vMotion in it's own None Routed Network

In the vSwitch properties --> Nic Teaming --> checked Overide vSwitch failover order:

In the Service Console Portgroup vmnic0 Active Adabter and vmnic1 Standby Adabter

In the vMotion Portrgoup vmnic1 Active Adabter and vmnic0 Standby Adabter

Rolling Failover Set to: Yes

Q: If the SC on it's own Adminitrative Segment "VLAN" and only IT and VC has access to it via ACL from Production Segment and vmnic0 the remaining nic is the vmnic1 which is dedecated to the vMotion Network. Is there any routing between vmnic1 and vmnic0 must be in place in order for the IT Administrators and VC reach the SC via vmnic1?

Best Regards,

Hussain Al Sayed

[Texiwill]

Hello,

No routing, remember you will be using VLANs, so 1 pNIC in a failover case will host both VLANs.

Also, place VC within the Admin network and I would also place workstations (VMs even) for the administrators in there, accessible from their desk or something via RDP through the Admin firewall. Otherwise you end up opening up quite a few ports to make things work and the other is much easier.

Best regards,

Edward L. Haletky

VMware Communities User Moderator

====

Author of the book 'VMWare ESX Server in the Enterprise: Planning and Securing Virtualization Servers', Copyright 2008 Pearson Education. CIO Virtualization Blog: http://www.cio.com/blog/index/topic/168354, As well as the Virtualization Wiki at http://www.astroarch.com/wiki/index.php/Virtualization