Not getting internet access

[flafleur]

Hi,

We recently started using NSX, but we are having some issues getting set-up. At first, we tried plugging two VMs behind a logical router, which itself had an uplink towards an Edge Gateway. The gateway doesn't have any problem to ping either internet or the VMs. The VM can ping each others and the dLR, but not the Gateway nor the internet. And, directly from the Router, we can't ping a thing.

Taking that last point into account, we decided to try any bypass the router altogether. We connected the VMs directly to the Gateway. This time, anything can ping anything, except for the fact that the two VMs can't ping the internet.

We tried looking around the internet, but everywhere we've looked it just seems to work and we can't really find the place we did something wrong.

Thanks in advance for your answer.

PS: both the Edges have a "Allow *" policy for the tests to make sure it was not the issue.

[Marcin4]

Hello,

How about routing protcolos ? have u configured static routes or dynamic routing protcolor form DLR to the EDGE ?

Best Regards

Marcin Gwóźdź

[flafleur]

I did a static route from the EDGE to the DLR so that it could know where its subnetwork was. On the router we tried enabling OSPF, but once enabled, it seems to drop every ping attempt, even to itself directly.

Forgot to say it explicitly, but OSPF is also on the EDGE

[Marcin4]

Do you have routes from DLR subinterfaces to -> EDGE/External network ?

[flafleur]

No, there are no routes defined on the DLR at the moment.

[Marcin4]

You need to define at least one static route from subnetwork of DLR to external network, so your vm's in VXLAN will know how to reach other networks.

Go to

1. Netowrking & Security -> NSX Edges

2. Click on your DLR logical router

3. Manage -> Routing

And now you have Two options:

A. Configure Dynamic routing with your NSX Edge

B. Configure Static routing so that your vm's on vxlan will know how to reach other networks.

Remember if you are using static routing in your network you need to creat that static router on all "devices" so each "Device" will know how to reach diferenet networks.

Could you please log in to EDGE and DLR and do the show ip route and post it here please ?

Best Regards

Marcin Gwóźdź

[flafleur]

I tried enabling BGP. I'll keep looking into it to make sure I set up everything correctly.

As for the route. From the DLR:

S     0.0.0.0/0        [1/0]     via 192.168.10.2
S     8.8.8.0/24       [1/0]     via 192.168.10.2
C     172.16.17.0/24   [0/0]     via 172.16.17.1
C     192.168.10.0/24  [0/0]     via 192.168.10.18

And from the Edge

S     0.0.0.0/0         [1/0]     via 54.38.107.158
C     54.38.107.144/28  [0/0]     via 54.38.107.145
S     172.16.17.0/24    [1/0]     via 192.168.10.17
C     192.168.10.0/24   [0/0]     via 192.168.10.2

Quick note :

     I configured the 8.8.8.0 static route, hoping it would help with the trafic, but it did not.

     Since adding the BGP, the edge is not able to ping the DLR not the VMs

Correction, the edge seems to be able to ping the VMs. I probably did not wait long enough for the configuration to spread

[Marcin4]

Allright if the edge is able to ping VM's inside DLR, it should work reverse way and VM should be able to ping Edge, can they ?

[flafleur]

They can without a problem. I still find it weird that the DLR can't ping a thing but, that aside, all the WMware side works well. The problem comes when I try to leave on the internet. the edge still pings the outside without a problem, but the VM can't. This was the main reason why I had removed the DLR at first as I assumed it was a miss-configuration I did on its routing protocol.

[Marcin4]

So the last thing is that VM's on VXLAN can not ping outside world for example 8.8.8.8  ??

[flafleur]

Exactly. As you can see in the routes earlier, I tried putting a static path for 8.8.8.0/24 from the DLR, hoping it would help, but it did not change a thing.

[Marcin4]

Could you post a tracerout detial doing from your VM that reside on VXLAN ?

I've made on my LAB an  exmaple tracerout from VM that reside on VXLAN for you.

Description:

1. DLR - Internal Interface

2. ESG - Internal Interface

3. Physical Router

4. Google

[mauricioamorim]

Have you checked your NAT device? Is it NATting the prefixes behind the DLR as well when going to the Internet?

[flafleur]

I mounted a OS with traceroute preinstalled, and here's the result :

[flafleur]

The NAT seems to be correct. I tried to remove the rules and recreate them just to make sure I did not do a typo

[Marcin4]

Where are you doing your NAT ? on EDGE or your physical router ?

Are you able to ping EDGE external interface(uplink to physical router) ?

Is the ECMP on EDGE on ?

[flafleur]

The NAT is down on the EDGE. The ping does work on both IP I gave the interface.

I did not enabled the ECMP

[Marcin4]

Any chance to draw the structure of your logic network, with all logical routers, logical switches, transport zones that you are using?

[flafleur]

Here's a quick overview. Note that we are on a cloud so the "internet" part include some router and switches handled by our hosting company

[flafleur]

Quick correction. The ECMP is enabled. I am not sure why but I was looking at something else earlier.