NSX-T 3.0 - tunnel down and PING not reachable to Tier 0

[rajkumar49]

hi,

we are building a single ESXi host based NSX-T 3.0 deployment.

we have two issues :

1. tunnel between the Host TEP and Edge transport node TEP is down.

2. As shown in the below topology diagram, from the Application server running in the segment1, we can ping the 10.204.253.18 IP, but unable to ping the external Tier-0 interface IP 10.204.253.191.

Please help.

Screenshots:

[rajkumar49]

screenshots:

[rajkumar49]

topology diagram :

[Sreec]

1.Can the Host Tep each Edge Tep ?

2.Do we have MTU set correctly ?

[rajkumar49]

1.Can the Host Tep reach Edge Tep ?

the Host main IP able to reach both Edge node management IP and Edge TEP IP.

2.Do we have MTU set correctly ?

i just modified vStandardSwitch to MTU 1600 , the VM Network in vSS is used in Edge uplink.

still the overlay tunnel is down. 

[rajkumar49]

here is the PING response from ESXi to :

[root@esxi:~] ping 10.204.253.46   (edge TEP ip)

PING 10.204.253.46 (10.204.253.46): 56 data bytes

64 bytes from 10.204.253.46: icmp_seq=0 ttl=64 time=4.137 ms

64 bytes from 10.204.253.46: icmp_seq=1 ttl=64 time=0.212 ms

64 bytes from 10.204.253.46: icmp_seq=2 ttl=64 time=0.284 ms

--- 10.204.253.46 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.212/1.544/4.137 ms

[root@esxi:~] ping 10.204.253.39 (Edge management IP)

PING 10.204.253.39 (10.204.253.39): 56 data bytes

64 bytes from 10.204.253.39: icmp_seq=0 ttl=64 time=0.456 ms

64 bytes from 10.204.253.39: icmp_seq=1 ttl=64 time=0.217 ms

64 bytes from 10.204.253.39: icmp_seq=2 ttl=64 time=0.259 ms

--- 10.204.253.39 ping statistics ---

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.217/0.311/0.456 ms

[Sreec]

Can you please test with MTU 1600 ?

[rajkumar49]

we have virtual switches with MTU 1600 . we use single ESXi host. whether the physical switch should have MTU 1600  for the tunnel between Edge node and host ?

[Sreec]

if host Tep and Edge Tep is in different VLAN , you need to set End-End MTU 1600 . I would appreciate if you can provide the output of my last ask ( MTU testing results) ?

[rajkumar49]

below are the MTU check results :

ESXi host 10.204.241.10 :

MTU is 1600 in both the N-VDS

MTU is 1600 in vmnic0,1,5

MTU is 1600 in the vmk10  TEP (IP - 10.204.253.38 )

Edge Transport Node 10.204.253.39 :

TEP Interface (IP- 10.204.253.46) has MTU 1600

Standard switch "vSwitch0" has the MTU 1600 - connected with Edge_node_1 (two port groups in the vSwitch0 are uplink to nvds1,nvds2 of the edge_node_1 )

PING : (from ESXi host)

To Edge node :

Command : vmkping ++netstack=vxlan 10.204.253.46 -d -s 1572  -I vmk10

Ping failed

To ESXi host:

Command : vmkping ++netstack=vxlan 10.204.253.38 -d -s 1572  -I vmk10

Ping success

[rajkumar49]

please find the MTU check results .

i followed the below steps :

https://spillthensxt.com/how-to-validate-mtu-in-an-nsx-t-environment/

[ax98]

You have to deal with the tunnel down problem first

First, whether the MTU meets the standard？

Second, whether the edge and the TEP IP of the host have separate subnets？

It is suggested that the TEP of edge and esxi should be separated from different subnets.

[p0wertje]

In a collapsed situation as you have it is required to have 2 subnets for the TEPS:

The VLAN/subnet for host overlay (TEP) and Edge VMs N-VDS overlay must be different and routing between host TEP and Edge-VM occurs at the physical layer, this requirement is coming from protecting the host overlay from VM generating overlay traffic.

Page 168 of VMware® NSX-T Reference Design