vmware networking Best Practice example?

[network_user]

Hello,

I am planning a new vmware two host cluster for the first time and I would like to get some guidance on the networking piece of it. Best practices suggest to separate vsphere management traffic from network traffic. Does that mean I should have a separate network/subnet for host management, vMotion, SAN and actual VM data traffic? And each of these should be on their own VLANS?

It would be easier for me to understand if I have an example with IPs and VLANS showed on them. Does anyone know of any good vmware best practices examples I can find?

Thank you.

Shivani

[HeathReynolds]

Chris Wahl has a new VMware networking book coming out in a couple of weeks, could be useful for you:

http://www.amazon.com/Networking-VMware-Administrators-Press-Technology/dp/0133511081/ref=sr_1_1?ie=...;

You are on the right track putting everything on it's own VLAN. You can then use the features of the vmware switch to manipulate the active, standby, and unused adapters for specific traffic types.

Do you have 10G or 1G networking? How many adapters per host? Do you have enterprise plus licensing available (required for distributed switch)?

[network_user]

Hello HeathReynolds,

Thanks for your reply. I have 10G networking , two hosts with 4 adapters per host. I dont think we have enterprise plus, but since there are nto many hosts, I guess standard switches should be fine.

So, you think if I design the network as below that should be the best practice?

Host Management : VLAN 100 -10.10.2.0/24

vMotion: VLAN 200 - 10.10.3.0/24

SAN/Storage: VLAN 300 - 10.10.4.0/24

VMs Traffic: VLAN 400 - 10.10.5.0/24

And use two physical NICs for VM traffic and other two for Host management, vMotion and SAN?

Or could it be done another better way?

[HeathReynolds]

Having four 10G interfaces gives you some flexibility. I'm assuming these aren't Cisco VIC cards or the broadcom cards that can create virtual partitions.

There is a section at the bottom of this cisco document that goes through some teaming options for the vSphere Standard Switch. May be a useful read, these are based around two 10G but should give you some ideas.

http://www.cisco.com/c/en/us/products/collateral/switches/nexus-1000v-switch-vmware-vsphere/white_pa...;

Your VLANs and IPs look fine. Your vMotion and SAN storage VLANs probably don't even have to be routable, you just need a L2 VLAL created on the upstream switch.

I would carve this up with two vSwitch instances, then manipulate the failover order for each vmKernel interface or port group.

MGMT (VLAN100) - vSwitch0 - Active vmnic0, standby vmnic1

vMotion (VLAN200) - vSwitch0 - Active vmnic0, standby vmnic1

Storage (VLAN300) - vSwitch0 - Active vmnic1, standby vmnic0

VMTraffic (VLAN400) - vSwitch1 - Active vmnic2, Active vmnic3

As you create new VM port groups on vSwitch1 they won't require editing, the defaults should be route based on port ID with both nics active. This will pin your VMs to on or the other active adapters based on port ID.

[network_user]

Hello HealthReynolds,

Thanks a lot for all the information and the guidance. This makes things very clear for me.

I have another question based on the design we discussed above. As per the above design we are using two NICs for 3 VLANS (MGMT, VMOTION and Storage). I tried to configure this on my test host and I got stuck at adding new port groups. I already have a management port group (VMKernel Port) on vSwitch0 and vmnic0 associated to it with network 10.10.2.0/24 with default GW as 10.10.2.1. Now when I try to add another port group for vMotion with different VLAN ID , it does give me an option to assign IP address (eg 10.10.3.2/24 to that port group, but its taking the default gateway as 10.10.2.1 by default. I am not sure how to assign a different default gateway for each VLAN. Logically i should have only one default gateway for a host. But in this case since I want to seggregate the traffic how do I achieve that? I think I am getting lost in the concepts somewhere or the actual configuration steps.

[HeathReynolds · Accepted solution]

You are on the right track. The ESX host only has one default gateway, and that default gateway should be on the MGMT vMkernel interface.

Other interfaces like vMotion and Storage (NFS and iSCSI) typically shouldn't be routed and don't need a default gateway. In 5.5 VMware introduced multiple TCPIP stacks and the ability to assign a VMkernel interface to a stack, but this functionality isn't needed for most installs.

You would simply place the vmotion interface of all of the hosts in your cluster in the same VLAN, and they will all be able to talk to each other without a L3 gateway on the VLAN. Same deal with you storage, put the hosts and the target interface of the storage on the same VLAN.

[network_user]

Hello HeathReynolds,

Thanks for all the help and guidance. I have a very clear understanding of what and how I need to do now. Thanks a million!

If I have any other questions, I may post a question again on this discussion.

Thanks!