- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
vMotion traffic isolation with VDswitch best practive question (ESXi 5.1)..
I have a question regarding what is the best practice for vMotion traffic utilizing a VDswitch in vSphere 5.1.
Our VDS is shared between two (soon to be 3) ESXi 5.1 hosts. They all have twelve 1GB physical NIC ports. I have created 11 port groups as follows:
- VLAN trunking is utilized.
- Four iSCSI port groups. They all physically lead to a switch that is isolated from other switches in our infrastructure so we isolate all iSCSI traffic. This is our VLAN 1060. These port groups each have a dedicated physical NIC assigned to them.
- All other port groups are physically connected to the same switch.
- One port group is for Fault Tolerance, VLAN 1066.
- One port group is for vMotion, VLAN 1065.
- One port group is for Management, VLAN 1100.
- We have four other port groups, each with their own VLAN ID. These are for VMs that will reside on various VLANs within our infrastructure (VLAN IDs 20, 30, 1100, 2003).
- All non-iSCSI port groups share the same active uplinks within the Teaming and Failover settings of the port group. Route is based on physical NIC load.
- We are using shares defined in the Network Resource Pool to prioritize traffic:
NFS=50, Management=5, vMotion =10, vSphere SAN=50, vSphere replication=50, iSCSI=50, VM=20, FT=10. I believe these are default values of the network resource pool.
Here is my question: Should we be isolating vMotion traffic by dedicating physical NICs that are exclusively dedicated for vMotion, rather than isolating via VLAN as I have described above? Will vMotion traffic degrade performance in the way I have it configured above?
From the various best practice white papers put out by VMware, I understand that vMotion traffic should be isolated from other traffic. I have done this by utilizing different VLANs for different traffic types. However, I am wondering if vMotion traffic should be isolated by using NIC ports dedicated exclusively for vMotion.
Any help with determining which design is best to use would be greatly appreciated.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Passing this along from one of our Engineers:
In a perfect world you'd want to do both of those things.
You want to isolate vMotion traffic in it's own broadcast domain - i.e. dedicated non-routable VLAN or a dedicated physical switch. And ideally, you'd want at least one dedicated physical network adapter for vMotion as it can be quite 'bursty' and may increase latency to other network services during migrations.
This may not always be possible - especially with blades where NIC counts are typically on the low side. Having vMotion isolated into it's own VLAN is the more important of the two considerations if you can't do both.
It sounds like you've done all of the reasonable remediation required to limit this problem though - with network resource pools, load based teaming etc.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You have a good setup and looks like it would work well.
I would reiterate the "perfect world" reference and say I tend to recommend to my customers that vMotion be seperated out if resources allow. In the case of a host with two 10G NIC's this is not possible so leveraging bi-directional traffic shaping features handle this. But in deployments where many NIC's are available, especially 1G NIC's, physically seperating vMotion traffic is my preference. Both get you there and both are good practices. I tend to choose segregation over software shaping simply because vMotion is very bursty and was designed to use as much bandwidth as it gets out of the box. In limited bandwidth scenarios (to me 1G NIC's are limited bandwidth) physically allocating bandwidth to vMotion takes care of all considerations in one swoop. Of course in high-bandwidth environments (multiple 10G NICs) it would be much better to carve up the bandwidth, let traffic shaping and control do its thing, and know that you are utilizing all of your bandwidth investment as efficiently as possible.
This is all relevent to the use case and context takes precident. Hope this helps - just my 2 cents. ![]()