1 2 Previous Next 21 Replies Latest reply on Jan 8, 2018 9:14 AM by Taupin

    Wrong BGP next hop programming

    Taupin Novice

      Hi,

       

      Please consider below topology:

       

      DLR1(AS65001)             --->           ESG1(AS65001)        ------------------------------> Physical Router(AS65002)

      192.168.1.1/30                    192.168.1.2/30                      192.168.100.3/24            192.168.100.1/24

       

      - Static route 172.16.0.0/19 is configured on DLR1 and redistributed via BGP to ESG1

      - ESG1 advertises stiatic route 172.16.0.0/29 to Physical Router

       

      - static route 172.16.0.0/19 is advertised from ESG1 to Physical router with 192.168.1.1 as Next Hop which is wrong and Physical router didn't install the static route

       

      I believe the Next Hop attribute programming is incorrect when ESG1 advertise the route to physical router, NEXT Hop must be 192.168.100.3 instead of physical Router IP address (ie 192.168.1.1)

       

      Please find attached output from ESG, the output includes:

      - Routes advertised to Physical router 192.168.100.1

      - Routes received from DLR1 : 192.168.1.1

      - directly connected routes on ESG1

       

      Best Regards

      Abdelfatah ELARFAOUI

        • 1. Re: Wrong BGP next hop programming
          Taupin Novice

          to reproduce the issue please consider the order of vaporization as below:

           

          - Creates static route then redistribute the static route

          or

          - Activate the redistribution and check static route/connected then create static route then clear bgp peer with the physical router

          • 2. Re: Wrong BGP next hop programming
            Taupin Novice

            Sorry for typo:

             

            Correct description:

             

            - Static route 172.16.0.0/19 is configured on DLR1 and redistributed via BGP to ESG1

            - ESG1 advertises stiatic route 172.16.0.0/29 to Physical Router

             

            - static route 172.16.0.0/19 is advertised from ESG1 to Physical router with 192.168.100.1 as Next Hop which is wrong and Physical router didn't install the static route

             

            I believe the Next Hop attribute programming is incorrect when ESG1 advertise the route to physical router, NEXT Hop must be 192.168.100.3 instead of physical Router IP address (ie 192.168.100.1)

             

            Please find attached output from ESG, the output includes:

            - Routes advertised to Physical router 192.168.100.1

            - Routes received from DLR1 : 192.168.1.1

            - directly connected routes on ESG1

            • 3. Re: Wrong BGP next hop programming
              Taupin Novice

              If I remove static route then recreate it, the NEXT HOP programming is correct and route is advertised to Physical Router with ESG1 IP address 192.168.100.3 Thus Route is installed on Physical router. Please check attached output from ESG1

               

              Once the clear bgp session with physical router, ESG1 will advertise the static route with 192.168.100.1 as NEXT HOP which is wrong

              • 4. Re: Wrong BGP next hop programming
                Bayu Wibowo Master
                vExpertCommunity WarriorsUser Moderators

                Hi, BGP next hop is not changed on iBGP sessions.

                If we refer to the VMware® NSX for vSphere Network Virtualization Design Guide ver 3.0, it has explanation on this topic too on page 68

                In eBGP/iBGP route exchange, when a route is advertised into iBGP, the next hop is carried unchanged into the iBGP domain.

                This may create dependencies on external routing domain stability or connectivity.

                To avoid external route reachability issues, the BGP next-hop-self feature or redistribution of a connected interface from which the next hop is learned is required.

                The BGP next-hop-self is not supported in current implementation, thus it is necessary to redistribute the ESG uplink interface (e.g., two VLANs that connect to physical routers) into the

                iBGP session towards the DLR. Proper filtering should be enabled on the ESG to make sure the uplinks’ addresses are not advertised back to physical routers as this can cause loops/failures.

                 

                The solution is to redistribute the ESG1 uplink 192.168.100.3/24 into BGP towards the DLR so DLR can reach the physical router 192.168.100.1

                 

                If you need more info on BGP around this specific topic, see these links

                BGP: Frequently Asked Questions - Cisco

                http://blog.ipspace.net/2011/08/bgp-next-hop-processing.html

                http://www.getnetworking.net/bgp/bgp-next-hop-self

                 

                Question, what is the requirement behind the static route on DLR1?
                If you are going to configure static route to summarise the logical switch networks behind DLR, this normally done on the ESG as per design guide

                Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                https://nz.linkedin.com/in/bayupw | twitter @bayupw
                • 5. Re: Wrong BGP next hop programming
                  Bayu Wibowo Master
                  vExpertCommunity WarriorsUser Moderators

                  There is probably a routing loop where the physical router advertise back the static route to the ESG or static route with that next hop.

                  Could you share your static routes and the route filtering configuration?

                  Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                  Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                  https://nz.linkedin.com/in/bayupw | twitter @bayupw
                  • 6. Re: Wrong BGP next hop programming
                    Taupin Novice

                    Hi,

                     

                    Please review my topology, ESG1-Physical router is an EBGP session!! so as per RFC and normal EBGP behavior NEXT HOP will be changed!

                     

                    Best Regards

                     

                    Abdelfatah ELARFAOUI

                    IP/MPLS Expert, CCIE R&S

                    • 7. Re: Wrong BGP next hop programming
                      Taupin Novice

                      There is no loop my friend, the topology is really a Flat topology!!

                       

                      I have included the procedure to reproduce this strange behavior which is not aligned with BGP RFC

                       

                      Best Regards

                       

                      Abdelfatah ELARFAOUI

                      IP/MPLS Expert | CCIE R&S

                      https://www.linkedin.com/in/elarfaouiabdelfatah/

                      • 8. Re: Wrong BGP next hop programming
                        Bayu Wibowo Master
                        vExpertCommunity WarriorsUser Moderators

                        Sorry I missed the ASN# detail in your first post. Which NSX version are you using? I'll try to simulate too in my lab

                        Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                        Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                        https://nz.linkedin.com/in/bayupw | twitter @bayupw
                        • 9. Re: Wrong BGP next hop programming
                          Bayu Wibowo Master
                          User ModeratorsCommunity WarriorsvExpert

                          I couldn't find the attachment on this reply

                          Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                          Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                          https://nz.linkedin.com/in/bayupw | twitter @bayupw
                          • 10. Re: Wrong BGP next hop programming
                            Taupin Novice

                            Hi,

                             

                            Thanks for your reply, NSX version 6.3.1

                             

                            To reproduce this issue please follow below procedure:

                             

                            - Topology:

                             

                            DLR2 (with default GW and directly connected subnets 172.16.1.0/24, 172.16.2.0/24,172.16.3.0/24) [192.168.1.1/30]----------> [192.168.1.2/30] DLR1 [192.168.13.1/24]-------------> [192.168.13.2/24]  ESG1 [192.168.100.3/24]---------------------------------------> [192.168.100.1/24] Physical Router

                             

                            - DLR2 uses default GW 0.0.0.0/0 pointing to DLR1 which in fact an ESG

                            - DLR1 and ESG1 are on the same AS 65001

                            - Physical Router on AS 65002

                             

                            - DLR1 is configured with summary route 172.16.0.0/21 pointing to DLR2

                            - DLR1 redistribute static route and directly connected routes to ESG1

                            - ESG1 redistribute directly connected routes

                             

                            To reproduce :

                             

                            - create static route Then redistribute the static route

                             

                            as Workaround: I am deleting the static route and recreate it then NEXT HOP is reprogrammed correctly by ESG1 but once I clear BGP sessions, the issue is reproduced

                             

                            Best Regards

                             

                            Abdelfatah ELARFAOUI

                            IP/MPLS Expert , CCIE R&S

                            https://www.linkedin.com/in/elarfaouiabdelfatah/

                            • 11. Re: Wrong BGP next hop programming
                              Bayu Wibowo Master
                              vExpertUser ModeratorsCommunity Warriors

                              Before I continue to test the scenario, are you saying that you have DLR2 behind DLR1?

                              Please note that building a multi-tier topology using only DLR instances is not supported and connecting multiple DLRs to a single ESG on shared VXLAN logical switch is also not supported

                              See VMware® NSX for vSphere Network Virtualization Design Guide ver 3.0 page 73 on Unsupported Topologies

                              Bayu Wibowo | vExpert NSX, VCIX6-DCV/NV, Cisco Champion, AWS-SAA
                              Author of VMware NSX Cookbook http://bit.ly/NSXCookbook
                              https://nz.linkedin.com/in/bayupw | twitter @bayupw
                              • 12. Re: Wrong BGP next hop programming
                                Taupin Novice

                                Please consider DLR1 as ESG,

                                 

                                Best Regards

                                Abdelfatah ELARFAOUI

                                • 13. Re: Wrong BGP next hop programming
                                  Taupin Novice

                                  Hi,

                                   

                                  Did you get the chance to simulate your Lab?

                                   

                                  Thanks

                                  • 14. Re: Wrong BGP next hop programming
                                    Taupin Novice

                                    Hi,

                                     

                                    Cloud you please share your finding, I can help with debug if you couldn't successfully reproduce the issue

                                     

                                    Thanks

                                    1 2 Previous Next