VMware Cloud Community
Kitty_Danvers
Contributor
Contributor

Nested Lab. vSphere 6.7. Promiscuous mode & MAC Learning confusion.

I am struggling to get my head round the networking for a nested environment.

I am seeing duplicate (DUP) packets when I PING between some nested VM's and I can see from ESXTOP ton my physical hosts that all my nested ESXi host VM's are getting the same traffic?

Here is my setup (apologies in advance).

Physical Lab

2 Physical ESXi Hosts (LAB-P-ESXI01 & 02) running version 6.7 U3 (b14320388).Each Host has 2 x 10Gb NICs, connected to a Cisco 10Gb Switch. All switch ports trunk/tag all VLANs. Static IP routing between VLANs in the switch.

1 Windows AD, DNS, DHCP Server (Used by both the physical and nested lab environments) – (LAB-V-MSAD01)

1 vCenter Server v6.7 U3b (b15129973) - (LAB-V-VCSA01)

A “Lab” vDS that is connected to both physical hosts.

“Lab” vDS Details

• vDS Version 6.6.0

• Uplink1 = vmnic1

• Uplink2 = vmnic2

• Various VLAN Port Groups and one VLAN Trunk Port Group

General

• Name = VLAN-Trunk

• Port binding = Static binding

• Port allocation = Elastic

• Number of ports  = 128

VLAN

• VLAN type = VLAN Trunking

• VLAN ID = 0-4094

Security

• Promiscuous mode = Accept

• MAC address changes = Accept

• Forged transmits = Accept

Teaming & Failover

• Load balancing = Route based on originating virtual port

• Network failure detection = Link status only

• Notify switches = Yes

• Failback = Yes

• Failover order = Active Uplinks: Uplink1, Uplink2

Nested Lab

1 vCenter Server v6.7 U3b (b15129973) – A different vCenter VCSA (LAB-V-VCSA02)

3 Nested ESXi VM’s (LAB-V-ESXI03, 04 & 05) running version 6.7 U3 (b14320388). Each nested host has 2 x VMXNET3 NIC’s. Both NIC’s are connected to the VLAN-Trunk Port Group. The ESXi host VM’s are built from William Lam’s “Nested ESXi Virtual Appliances”, so include the required ‘dvFilter Mac Learn VMX params’.

A “Nested-Lab” vDS that is connected to the 3 nested hosts.

“Nested-Lab” vDS Details

• vDS Version 6.6.0

• Uplink1 = vmnic1

• Uplink2 = vmnic2

• Various VLAN Port Groups (Management, vMotion, vSAN, VM’s): Example

General

• Name = VLAN15-Mgmt

• Port binding = Static binding

• Port allocation = Elastic

• Number of ports  = 128

VLAN

• VLAN type = VLAN

• VLAN ID = 15

Security

• Promiscuous mode = Reject

• MAC address changes = Reject

• Forged transmits = Reject

Teaming & Failover

• Load balancing = Route based on originating virtual port

• Network failure detection = Link status only

• Notify switches = Yes

• Failback = Yes

• Failover order = Active Uplinks: Uplink1, Uplink2

I know the MAC Learning is native in vSphere 6.7, so that I no longer need to install the “ESXi Mac Learning dvFilter” Fling, but I am not clear on exactly on what else I need to do?

I see that William Lam also created a couple of PowerCLI functions (Get-MacLearn & Set-MacLearn), but on which vDS/Port Groups do I enable MAC Learning? The “Lab” or the “Nested-Lab” or both?

And once I have enabled MAC Learning on the correct vDS/PG, do I need to change the 'legacy' Security settings on either of the “Lab” or the “Nested-Lab” vDS/PGs?

If I upgrade the Nested Lab environment to vSphere 7, does anything change? (I cannot upgrade the physical ESXi hosts to v7.0, as I run SanDisk FusioIO PCIe Flash storage; which does not work with v7.0).

Thanks

Kitty

Tags (3)
Reply
0 Kudos
2 Replies
MinoDC
Enthusiast
Enthusiast

Hi Kitty...I've same questions....do you have found the answer ?

Thanks...

Reply
0 Kudos
mabrown76
VMware Employee
VMware Employee

This KB will address your issue. https://kb.vmware.com/s/article/59235

 

Basically on the physical host(s) you need to run 

esxcli system settings advanced set -o /Net/ReversePathFwdCheckPromisc -I 1

Then disable and re-enable promiscuous mode on all your port groups.

--
Mike Brown, VCDX² (DCV & NV)
Senior Staff Architect
@vcdx71
Reply
0 Kudos