communication between port group and vmkernel in the same vlan requires promiscuous mode

[realdreams]

The networking on the hostC is set up like this. vmnic 0 is trunked to the switch

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I have a nested ESXi VM named ESXi-Test1 on hostC.

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Guest VM ESXi_Test1's vMotion uses vmnic5, which is in VM Network vMotion port group with VLAN 3. The host's vMotion vmkernel also uses VLAN 3. But the problem is with promiscuous mode disabled, I can ping vMotion VMkernel for hostC but not vMotion VMkernel for guest ESXi(ESXi_Test1). And they cannot ping each other and of course vMotion fails.

Once I have promiscuous mode enabled, they can ping each other and vMotion works.

This is true for all port groups I am using. On the same vSwitch, a port group and a vmkernel with the same VLAN ID cannot communicate with each other with promiscuous mode disabled. IMO this communication does not violate VLAN policies(in the same VLAN). Did I set something up incorrectly or VMWare implemented vSwitch this way for some specific purpose? How can I allow communication between a port group and a vmkernel with the same VLAN ID if I don't want promiscuous mode enabled? It can cause both security and performance issues IMO.

[a_p_]

I'm afraid you won't get this to work without promiscious mode enabled. The outer host's vSwitch does not know about the MAC addresses of the inner host's VMKernel ports/VM's and therefore will not forward any traffic to them unless promiscuous mode is enabled.

André

[realdreams]

Why the outer switch can't properly learn the mac addresses?

They are in the same VLAN. Every NIC/VM Kernel in the port group is "directly" connected to one switch port. What breaks ARP resoluiton?

[a_p_]

Technically it would certainly be possible to make a vSwitch learn the MAC addresses, but this is not implemented. Please take a look at e.g. http://kensvirtualreality.wordpress.com/2009/03/29/the-great-vswitch-debate-part-2/ (Miscellaneous Security Features) for details.

André

[realdreams]

If the guest connected to the port group is a regular OS(e.g. Win 2K8 R2) it works just fine. Why ESXi does not work?

Does this have something to do with the guest's vmnic? ESXi's vmnics are always in trunk mode but the port group is in access mode?

[a_p_]

The reason it doesn't work is the cascading of vSwitches. The MAC addresses of the innner ESXi (VMKernel and VMs) are only known to the inner host's vSwitch to which they are directly connected.

André