VMware Cloud Community
hccvm
Contributor
Contributor

vMotion MAC Address Flapping

We have a vDS set up with default settings and are seeing an increased amount of port flapping that happens whenever a VM is migrated to another host in the cluster by DRS. Is there any option for the vDS that can alleviate this, or is this expected behavior because of the vMotion?

Thanks.

Reply
0 Kudos
7 Replies
Lalegre
Virtuoso
Virtuoso

Hey hccvm​,

How is your teaming and failover and security configuration in the vMotion portgroup and the VM portgroup?

Reply
0 Kudos
hccvm
Contributor
Contributor

Hi Lalegre,

Below are the portgroup settings:

Vmotion Portgroup

Security

Promiscuous Mode: Reject

Mac Address Changes: Accept

Forged Transmits: Accept

Failover and Teaming

Load Balancing: Route based on originating virtual port

Network Failure Detection: Link Status Only

Notify Switches: Yes

Fallback: Yes

VM Portgroup

Security

Promiscuous Mode: Reject

Mac Address Changes: Reject

Forged Transmits: Reject

Failover and Teaming

Load Balancing: Route based on IP Hash

Network Failure Detection: Link Status Only

Notify Switches: Yes

Fallback: Yes

Reply
0 Kudos
Lalegre
Virtuoso
Virtuoso

Hey,

First of all i can see that on one portgroup you are using Route based on IP Hash and on the other one you are using Route based on originating virtual port. Here i can assume that is because it is a vMotion portgroup and you only one have one active uplink. Route based on IP Hash should be always used with LAG in your physical switches configured.

Then on the vMotion portgroup i can see this:

Mac Address Changes: Accept

Forged Transmits: Accept

Could you please try to change to Reject on both and try again to see if the issue still persist?

Reply
0 Kudos
depping
Leadership
Leadership

Make sure that your physical network configuration aligns with the virtual. In other words, as mentioned above, if you want to use "IP Hash" you need to make sure you have an etherchannel. Also from a networking point of view make sure spanning tree is correctly configured, this is described here: VMware Knowledge Base .

Reply
0 Kudos
hccvm
Contributor
Contributor

Thank you Duncan for the KnowledgeBase article, STP is configured as indicated there.

Regarding the IP Hash on the port group, I'm not sure if it is because it is a default setting, but what should be used if there is no Etherchannel on the physical port? The switches that the ESXi hosts are connected to are themselves using Etherchannel but the ESXi hosts only have 1 VM network uplink and no Etherchannel.

I will get another report from our networking team to see if lalegre's suggestion has reduced the flapping. But is flapping to be expected during vMotion or is it an indication that something is misconfigured? The ESXi hosts in the cluster are connected to different data centers.

Thanks

Reply
0 Kudos
hccvm
Contributor
Contributor

So I used two sample VMs to test. One had "IP Hash" and the other had Route on source Virtual Port on different poirtgroups. The theory was the IP Hash one would have mac flapping, the other wouldn't. Unfortunately, both still had flapping while vMotion.

I don't see any other setting in vSphere that could be misconfigured to be causing this. The VM network is on each host on only 1 trunk uplink per host, no teaming. The vMotion network has 2 uplinks per host, no teaming and a separate vlan from VM network.

Reply
0 Kudos
depping
Leadership
Leadership

Just use Route based on Virtual Port ID. IP Hash should only be used for when you have etherchannels configured. In the past we have seen issues with mac flapping also when Beacon Probing was enabled. So I would also recommend to check that.

Reply
0 Kudos