2 Replies Latest reply on Feb 28, 2020 12:01 PM by mvrk

    NSX - unable to communicate from physical ESXi to nested ESXi's

    mvrk Enthusiast

      Hi,

       

      I'm having the same problem as described on this post: https://communities.vmware.com/message/2929755#2929755

      but as its a very old post, i decided to open a new one.

       

      On the blog mentioned on that previous post: https://telecomoccasionally.wordpress.com/2016/03/10/from-the-dept-of-the-knowledge-arcane-nsx-v-with-nested-esxi/

      solutions 1,2,3 are not an option for me, but one guy on that blog talks of another solution, about running nested VXLAN transport on VXLAN portgroup instead of VLAN as a solution, anyone knows exactly what he means by that? How to implement that?

       

      This is my current layout:

       

      vCenter 6.7

      ESXi's 6.5

      NSX 6.4.6

       

      Physical ESXi

      DS (mtu 1600)

        DPG-LAN (vlan 1) - uplink 1 - physical nic

        DPG-LAN-NESTED (vlan trunk / accepts Promiscuous mode, MAC changes, Forged transmits) - uplink 1 - physical nic

        VXLAN configured on vlan 4

       

      VM-TST0 - vnic with with virtual network on ...virtualwire-1-sid-5000 of distributed switch DS

       

       

      Nested ESXi1 (uses DPG-LAN-NESTED as virtual network) / Nested ESXi2 (uses DPG-LAN-NESTED as virtual network)

      DS-TST (mtu 1600)

        DPG-TST-LAN (vlan 1) - uplink 1 - virtual nic

        VXLAN configured on vlan 4

       

      VM-TST1 - vnic with with virtual network on ...virtualwire-1-sid-5000 of distributed switch DS-TST

      VM-TST2 - vnic with with virtual network on ...virtualwire-1-sid-5000 of distributed switch DS-TST

       

       

      NSX Manager and NSX Controllers running on physical ESXi.

       

       

      ping ++netstack=vxlan -d -s 1572 ... between all the 3 vxlan IPs work fine.

       

      On the Logical Switch (virtualwire 5000) VXLAN Standard Host-Ping test works between the 2 nested ESXi's but fails between physical ESXi and nested ESXi's.

       

      I can ping between VM-TST1 and VM-TST2 (nested ESXi's) but i can't ping from VM-TST0 (physical ESXi) to VM-TST1 or VM-TST2 (nested ESXi's).

        • 1. Re: NSX - unable to communicate from physical ESXi to nested ESXi's
          Sreec Master
          Community WarriorsVMware EmployeesvExpert

          Basically you can achieve nested ESXI with VXLAN traffic to physical host were you have VM's  by having a VXLAN portgroup connected to the Nested ESXI VM ( As of now you are leveraging DPG-LAN-NESTED (vlan trunk / accepts Promiscuous mode, MAC changes, Forged transmits) - uplink 1 - physical nic ) .  Also ensure MTU is 1600/9000 end to end..

          So it would like this Physical Host- VXLAN Network (Logical Switch) connected to  Nested ESXI , VM Connected to VXLAN Network (Nested VXLAN portgroup from a overall architecture perspective)

          However by far the best option is , keep physical host in VLAN only setup ( No NSX preparation) .

          • 2. Re: NSX - unable to communicate from physical ESXi to nested ESXi's
            mvrk Enthusiast

            Thanks.

             

            Anyway, today i was able to free one port in my home router and connected there the 2nd nic of my physical ESXi, so i went for solution 1 of the blog (separated the physical and nested ESXi's in different dswitche's).

             

            Everything working fine now.