NSX-T 3.1 North-South routing

[emistery]

Hi all,

I'm pretty new to NSX-T, and trying to set up a lab to gain some knowledge about it. I already set up some segments with VMs connected to it, a T1 router and a T0 router. The VMs can reach the uplink interface of the T0 router, but no further (the physical router). What are some troubleshooting steps I could try to do? All suggestions or tips are welcome.

[emistery]

Can everything still work properly with the DUP pings?

Also, I just reconfigured the segment to be connected to the T1 router, and I can't even ping other machines in the same segment anymore.. Looks like there is something else going on as well.

[shank89]

It should be Ok for a nested environment. 

What about 2 VM's, on the same segment on the same host?  If that works, try e/w pining from one host vmkping ++netstack=vxlan <IPohanotherhostTEP> -s 1572 -d, if that doesn't work you have an MTU issue between the hosts.  Then re-attempt this but the destination set as the edge TEP and check the results there.

[emistery]

Thanks for all the amazingly fast responses!

2 VMs on same segment on the same host won't work. The vmkping from host to host works fine, from host to edge works but give DUP.

I'm kinda stuck on the VM part as it should be very basic and it should work like this..

[shank89]

Hmm if the VM's can't get to each other on the same segment then there is likely some other issue, can they hit their own gateway?

No problems for the quick response.

[emistery]

No, they can't reach their gateway. Tried it on a T1 and a T0 router.

[shank89]

Subnets on the VMs configured correctly?  

[shank89]

log onto the host using SSH;

- type nsxcli

- get logical-router (can you see the t1 gateway?)

- get logical-switches (is the one your VMs attached to showing up?

[emistery]

Yes it sees the T1 GW and the logical switches that are connected.

Here the conf of the subnet:

Machine 1:

App01 machine

Machine 2:

App03 machine

NSX-T segment:App segment

[shank89]

Same issue occurring on other hosts?

[emistery]

Yes, cannot reach gateway and other machines on the same subnet on different hosts.

[shank89]

It might be worthwhile doing some packet captures to see what is going on, this may be of use https://spillthensxt.com/nsx-t-data-path-troubleshooting-using-nsxcli-capture/

Also go through and verify your config, if you are unsure post screen shots of the profiles you configured, segments created, t0 / t1's created.

[emistery]

Wow, according to traceflow everything gets delivered between VMs. A lot of duplicates, but that is normal apparently. Also, traceflow registers some old IP addresses of the machines as the current IP address. I reused some machines and put them in the nsx-t segment, and changed IP addressess. But I guess since traceflow registers everything normally, a normal ping should work right?

[shank89]

The nested stuff does complicate things a little, but yea pings should work.  No firewalls or anything?

[emistery]

Nope. Looks all very strange to me..

[shank89]

Hmm, on the host, go back to nsxcli, dig into the logical router and see if it has a downlink interface with the gateway IP set?

[emistery]

How can I see the interfaces on a logical router on an esxi host?

[shank89]

get logical-router <uuid> interfaces

[emistery]

The tier0 router:

The tier1 router:

[shank89]

get logical-switches <switch> mac-table - do you see mac addresses of your VMs?

[emistery]

Yes, the mac addresses of the VMs that are located on that host are visible under local, the others are under remote.