1 2 Previous Next 23 Replies Latest reply on Jun 15, 2019 4:14 AM by Rob80

    Static route issues on NSX-T

    Rob80 Novice

      Hi all,

       

      It seems that I can't find the way correctly connect Tier-0 to external network via static route on NSX-T 2.4. I have attached below my provisional network setup and I can't understand where the issue occur. Any help would be appreciated.

       

      Thanks

        • 1. Re: Static route issues on NSX-T
          daphnissov Guru
          Community WarriorsvExpert

          Your diagram shows you have two separate uplinks from your edges on VLAN overlays to two different networks. Not sure why you're doing this as it doesn't make too much sense. The proper way to do this is configure those uplinks on your T0 to be in the same subnet. One port per edge. Configure a HA VIP across those two T0 uplinks. Your static route on your Meraki has its next hop set to that HA VIP. However, in order to route outbound traffic from NSX-T land, you will need to configure a default route (0.0.0.0/0) on your T0 directed to the next hop upstream. That is usually on the same subnet as the T0 uplink in the VLAN overlay.

          • 2. Re: Static route issues on NSX-T
            Rob80 Novice

            Daphnissov,

             

            Currently I have setup each uplink on the same subnet and connected to one port per edge as described above but would appreciate guidance regarding HA VIP configuration.

             

            Thanks

            • 3. Re: Static route issues on NSX-T
              daphnissov Guru
              vExpertCommunity Warriors

              This should be covered in the official docs. You configure the HA VIP on your T0. After creating uplinks for each edge, you create the VIP but do not assign it to any interface. Leave the field blank and it will auto-assume both available uplinks. Goes without saying but the HA VIP needs to be in the same subnet as the individual uplink ports.

              • 4. Re: Static route issues on NSX-T
                Rob80 Novice

                I did search online and number of times on NSX-t 2.4 and can't find HA VIP configuration only router ports.

                 

                • 5. Re: Static route issues on NSX-T
                  daphnissov Guru
                  Community WarriorsvExpert

                  Configuration => HA VIP

                   

                   

                   

                   

                   

                  • 6. Re: Static route issues on NSX-T
                    IPv4toIPv6 Lurker
                    VMware Employees

                    Hello Rob80,

                     

                    Base by you PDF you have 2 uplinks in 2 different vlans.

                    Did you check if you have L2 connectivity

                    VLAN 2011 from MX65 Meraki --- Edge Uplink1

                    Example:

                    MX65 Meraki  IP 192.20.11.1

                    Edge Uplink1 IP 192.20.11.2

                    There suppose to be connectivity.

                    Ping form 192.20.11.1 to 192.20.11.2 suppose to be ok

                    VLAN 2012 from MX65 Meraki --- Edge Uplink2

                    MX65 Meraki  IP 192.20.12.1

                    Edge Uplink1 IP 192.20.12.2

                    There suppose to be connectivity.

                    Ping form 192.20.12.1 to 192.20.12.2 suppose to be ok

                     

                    Please before going to the Routing configuration check the steps from above to be sure you have L2 connectivity

                     

                    Then go on edge routing and add destination 0.0.0.0/0  GW 192.20.11.1,192.20.12.1

                    And if you have a default route just delete'it.

                    My recommendation is still to use BGP

                    • 7. Re: Static route issues on NSX-T
                      Rob80 Novice

                      Hi,

                       

                      HA VIP has now been configured with IP of 192.20.11.5 which I can ping from VM's.

                      I added the following addresses to nsx-t  static route

                       

                      And another IP on MX

                       

                      But still no connection to outside world.

                      • 8. Re: Static route issues on NSX-T
                        Rob80 Novice

                        Hi,

                        In order to use bgp I need router capable in supporting bgp as one of the prerequisites is AS which seems can be only setup when mx in vpn concentrator mode, but then I have no vlans or network.

                         

                        Thanks

                        • 9. Re: Static route issues on NSX-T
                          daphnissov Guru
                          vExpertCommunity Warriors

                          Ok, I need to be able to see these screenshots you're posting as they're extremely small. Please also show your edge profiles and what interfaces are connected where. Also show your transport zone (overlay) profile and how your hosts are connected.

                          • 10. Re: Static route issues on NSX-T
                            Rob80 Novice

                            Please find attached few screen shots of existing configuration.

                            • 11. Re: Static route issues on NSX-T
                              daphnissov Guru
                              Community WarriorsvExpert

                              Ok, this gives me a better idea. Here are questions for you to check out based on your images.

                               

                              1. Your overlay profile is specifying VLAN 1020. This means the actual vmnics you're defining on your ESXi hosts are connected to upstream ports in trunk mode with 1020 allowed. Are you certain this is the case? If yes => Ping something else on this VLAN. Remember to pass the -S flag to vmkping to use the VXLAN TCP/IP stack. If no => set to proper VLAN. Use VLAN 0 if connected to an access port, or if this transport node (ESXi host) is virtual and connected to a virtual switch (in which case the VLAN tag is stripped off).
                              2. Your edge uplink profile profile is specifying an MTU of 1600. Normally this is left at 1500. Are you certain that you have MTU 1600 available on the VLANs in use by your edge uplinks to the physical infra? If yes => You're ok, but verify. If no => Change to 1500. To eliminate this variable, I'd change to 1500 regardless for testing.
                              3. Your T0 uplinks on the edges are in 192.20.11.0/24 with addresses of .2 and .3 for the two edges. What's the HA VIP address? What is the gateway address on this segment?
                              4. You are mixing configurations in the new policy-based API of 2.4 and the older format of 2.3. Anything you have under Networking => Tier-0 Gateways please remove. All configuration should be under Advanced Networking & Security.
                              5. From Advanced Networking & Security => Routers => T0, show Routing => Static Routes
                              6. Show from the same menu Route Redistribution
                              7. Show your edge cluster
                              8. Show your T1 => Configuration => Router Ports
                              9. Show your T1 => Routing => Route Advertisement
                              • 12. Re: Static route issues on NSX-T
                                Rob80 Novice

                                I have attached file with screens to some of the lines below.

                                 

                                In terms of Tier-0, do I need to remove all and recreate on Advanced networking? Shall I do the same with Tier-1 or keep as it is.

                                 

                                Gateway for HA VIP is subnet on meraki router 192.20.11.1

                                • 13. Re: Static route issues on NSX-T
                                  daphnissov Guru
                                  vExpertCommunity Warriors

                                  Your vmkping command in your screenshot is not what I wrote. Flags are case sensitive. From the ESXi host, do it again and check the result:

                                   

                                  vmkping -S vxlan <TEP> -d -s 1572 -c 10

                                  From your ESXi host, you should be able to do a vmkping from the TEP to the:

                                   

                                  • TEP of your edge transport nodes (both of them) on the same subnet

                                   

                                  From a VM that is attached to a logical switch, you should be able to ping (do not alter MTU):

                                  • Gateway (T1 downlink for this segment)
                                  • T1 linked port to T0
                                  • T0 uplink HA VIP address
                                  • Gateway on the HA VIP network (which you say should be 192.20.11.1)

                                   

                                  After you've corrected all these things, please indicate which of these ping tests pass and which fail.

                                   

                                  Regarding your static route, it's wrong. The T0 static route needs to be the gateway for that segment which is the SVI on the Meraki. So it needs to be 192.20.11.1.

                                   

                                  Your Meraki must have a static route setting the next hop for any networks you do not wish to NAT as the HA VIP address.

                                   

                                  In terms of Tier-0, do I need to remove all and recreate on Advanced networking? Shall I do the same with Tier-1 or keep as it is.

                                  Yes, to eliminate complexity, remove anything you have for routing and switching that's not under Advanced Networking & Security.

                                  • 14. Re: Static route issues on NSX-T
                                    Rob80 Novice

                                    NSX-T to edge node ping didn't succeed:

                                     

                                    -- 192.20.11.5 ping statistics ---

                                    10 packets transmitted, 0 packets received, 100% packet loss

                                     

                                    [root@ESXi:~] vmkping -S vxlan 192.20.11.5 -d -s 1572 -c 10

                                    PING 192.20.11.5 (192.20.11.5): 1572 data bytes

                                     

                                    --- 192.20.11.5 ping statistics ---

                                    10 packets transmitted, 0 packets received, 100% packet loss

                                     

                                    [root@ESXi:~] vmkping -S vxlan 192.20.11.2 -d -s 1572 -c 10

                                    PING 192.20.11.2 (192.20.11.2): 1572 data bytes

                                     

                                    --- 192.20.11.2 ping statistics ---

                                    10 packets transmitted, 0 packets received, 100% packet loss

                                     

                                    [root@ESXi:~] vmkping -S vxlan 192.20.11.3 -d -s 1572 -c 10

                                    PING 192.20.11.3 (192.20.11.3): 1572 data bytes

                                     

                                    --- 192.20.11.3 ping statistics ---

                                    10 packets transmitted, 0 packets received, 100% packet loss

                                     

                                    Regarding below

                                    • Gateway (T1 downlink for this segment) - success
                                    • T1 linked port to T0 - success
                                    • T0 uplink HA VIP address - success
                                    • Gateway on the HA VIP network (which you say should be 192.20.11.1) - unsuccessful

                                     

                                    1 2 Previous Next