I am struggling to get my head round the networking for a nested environment.
I am seeing duplicate (DUP) packets when I PING between some nested VM's and I can see from ESXTOP ton my physical hosts that all my nested ESXi host VM's are getting the same traffic?
Here is my setup (apologies in advance).
Physical Lab
2 Physical ESXi Hosts (LAB-P-ESXI01 & 02) running version 6.7 U3 (b14320388).Each Host has 2 x 10Gb NICs, connected to a Cisco 10Gb Switch. All switch ports trunk/tag all VLANs. Static IP routing between VLANs in the switch.
1 Windows AD, DNS, DHCP Server (Used by both the physical and nested lab environments) – (LAB-V-MSAD01)
1 vCenter Server v6.7 U3b (b15129973) - (LAB-V-VCSA01)
A “Lab” vDS that is connected to both physical hosts.
“Lab” vDS Details
• vDS Version 6.6.0
• Uplink1 = vmnic1
• Uplink2 = vmnic2
• Various VLAN Port Groups and one VLAN Trunk Port Group
General
• Name = VLAN-Trunk
• Port binding = Static binding
• Port allocation = Elastic
• Number of ports = 128
VLAN
• VLAN type = VLAN Trunking
• VLAN ID = 0-4094
Security
• Promiscuous mode = Accept
• MAC address changes = Accept
• Forged transmits = Accept
Teaming & Failover
• Load balancing = Route based on originating virtual port
• Network failure detection = Link status only
• Notify switches = Yes
• Failback = Yes
• Failover order = Active Uplinks: Uplink1, Uplink2
Nested Lab
1 vCenter Server v6.7 U3b (b15129973) – A different vCenter VCSA (LAB-V-VCSA02)
3 Nested ESXi VM’s (LAB-V-ESXI03, 04 & 05) running version 6.7 U3 (b14320388). Each nested host has 2 x VMXNET3 NIC’s. Both NIC’s are connected to the VLAN-Trunk Port Group. The ESXi host VM’s are built from William Lam’s “Nested ESXi Virtual Appliances”, so include the required ‘dvFilter Mac Learn VMX params’.
A “Nested-Lab” vDS that is connected to the 3 nested hosts.
“Nested-Lab” vDS Details
• vDS Version 6.6.0
• Uplink1 = vmnic1
• Uplink2 = vmnic2
• Various VLAN Port Groups (Management, vMotion, vSAN, VM’s): Example
General
• Name = VLAN15-Mgmt
• Port binding = Static binding
• Port allocation = Elastic
• Number of ports = 128
VLAN
• VLAN type = VLAN
• VLAN ID = 15
Security
• Promiscuous mode = Reject
• MAC address changes = Reject
• Forged transmits = Reject
Teaming & Failover
• Load balancing = Route based on originating virtual port
• Network failure detection = Link status only
• Notify switches = Yes
• Failback = Yes
• Failover order = Active Uplinks: Uplink1, Uplink2
I know the MAC Learning is native in vSphere 6.7, so that I no longer need to install the “ESXi Mac Learning dvFilter” Fling, but I am not clear on exactly on what else I need to do?
I see that William Lam also created a couple of PowerCLI functions (Get-MacLearn & Set-MacLearn), but on which vDS/Port Groups do I enable MAC Learning? The “Lab” or the “Nested-Lab” or both?
And once I have enabled MAC Learning on the correct vDS/PG, do I need to change the 'legacy' Security settings on either of the “Lab” or the “Nested-Lab” vDS/PGs?
If I upgrade the Nested Lab environment to vSphere 7, does anything change? (I cannot upgrade the physical ESXi hosts to v7.0, as I run SanDisk FusioIO PCIe Flash storage; which does not work with v7.0).
Thanks
Kitty
Hi Kitty...I've same questions....do you have found the answer ?
Thanks...
This KB will address your issue. https://kb.vmware.com/s/article/59235
Basically on the physical host(s) you need to run
esxcli system settings advanced set -o /Net/ReversePathFwdCheckPromisc -I 1
Then disable and re-enable promiscuous mode on all your port groups.