Hi
First very new to NSX-T, so excuse me if I use the incorrect terminology. I have roughly followed this setup -> Tier-0 Gateway - The IT Hollow
The issue I am seeing is, if I have no VM's connected to any of the segments, I am able to ping the segment gateways. The moment I connect a VM to any segment all communications to the segment gateways goes down. When I look at the Transport Zones my overlay TZ shows degraded. The VM connection has cause the edge and the host on which the VM resides to go down. I have a single static route on my T0 of 0.0.0.0/0 with the next hop being my physical switch. Any idea what I have done wrong or might be missing.
Running NSX-T Version 3.1.0.0.0.17107209
Hi,
Are you using physical switches ? If so, what mtu is configured on them ?
The physical switch is set for 9000. The DVS is set to 1600. From the ESXi hosts I have confirmed that I can ping using a 1572 packet size.
When you click the "down" does it give you any extra information ?
You could go look on the esx host and use 'nsx-cli' command
get host-switch <switch-name> tunnel
get host-switch <switch-name> tunnels <x.x.x.x> <y.y.y.y>
get logical-switches
get logical-switch <logical-switch id> vtep-table
Might give you more info on why the tunnel is down.
There is no information related to why things are in a down state.
Below is the output of some of the commands you mentioned.
host1> get host-switch Homelab-DVS tunnel
Sun May 02 2021 UTC 21:21:15.101
Local IP Remote IP Local State Remote State
192.168.202.1 192.168.202.253 down down
192.168.202.1 192.168.202.252 down down
host1> get host-switch Homelab-DVS tunnel 192.168.202.1 192.168.202.253
Sun May 02 2021 UTC 21:22:38.262
Local State : down
Remote State : down
Local Diag : No Diagnostic
Remote Diag : No Diagnostic
minRx : 1000
minTx : 1000
Local Disc : 0x7cdcdc5a
Remote Disc : 0x0
Tx Interval : 1000
Rx Interval : 1000
mult : 3
host1> get logical-switches
Sun May 02 2021 UTC 21:24:49.646
Logical Switches Summary
------------------------------------------------------------
Overlay Kernel Entry
============================================================
VNI DVS name VIF num
73734 Homelab-DVS 2
73738 Homelab-DVS 1
73735 Homelab-DVS 1
73736 Homelab-DVS 1
73737 Homelab-DVS 1
Overlay LCP Entry
============================================================
VNI Logical Switch UUID Name
73734 25af8478-36c1-4f0e-9b69-236d4fa9d2f5 SEG-1
73738 e7113bad-dfb4-4f78-bd84-64fc037ca8a1 SEG-4
73736 50f8e019-2188-41bb-b7ee-57eec010a63d SEG-2
73737 ae4e8c2b-4c76-43f1-bb2b-2d6fa3d9a719 SEG-3
73735 aa54ddaf-6536-46fc-8e92-65b1c7b7bf78 transit-bp-d9a418b3-317b-4108-b11b-67d123d9a027
VLAN Backed Entry
============================================================
Logical Switch UUID VLAN ID
host1> get logical-switch 25af8478-36c1-4f0e-9b69-236d4fa9d2f5 vtep-table
Sun May 02 2021 UTC 21:25:52.246
Logical Switch VTEP Table
-----------------------------------------------------------------------------------------------
Host Kernel Entry
===============================================================================================
Label VTEP IP Segment ID Is MTEP VTEP MAC BFD count
LCP Remote Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
LCP Local Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
114695 192.168.202.1 192.168.202.0 00:50:56:61:17:7b None
host1> get logical-switch aa54ddaf-6536-46fc-8e92-65b1c7b7bf78 vtep-table
Sun May 02 2021 UTC 21:26:09.253
Logical Switch VTEP Table
-----------------------------------------------------------------------------------------------
Host Kernel Entry
===============================================================================================
Label VTEP IP Segment ID Is MTEP VTEP MAC BFD count
114691 192.168.202.252 192.168.202.0 0 00:50:56:95:00:1a 0
LCP Remote Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
114691 192.168.202.252 192.168.202.0 00:50:56:95:00:1a fp-eth1
LCP Local Entry
===============================================================================================
Label VTEP IP Segment ID VTEP MAC DEVICE NAME
I have a single edge, with fp-eth0 and fp-eth1 connected to a trunked DVS port group. eth0 is connected to a DVS port group with no VLAN. My edge and my hosts are using the same uplink profile, which has a Teaming policy of Load Balance Source. It does not make sense to me that I am able to ping segment GW when no VM is connecte, then the moment it gets connected things go south.
- does adding a vm on host1 and a vm on host2 and ping between them (in the same segment) work ?
- do you have a VM based edge ? If so, are you using the same ip range for the teps ? if so, you need to create a nsx-t vlan backed segment
How did this one end up? Sitting in front of the same problem, but using Nested ESXi using non-nested NSX-T segments.
The moment I connect a VM to the segment, it goes down. TEP are pingable though.
I could never get to the bottom of that issue and ended up destroying my NSX-T lab. I have since rebuilt it using the details in this blog
NSX-T 3.1 Home lab - Setup and Fabric configuration | rudimartinsen.com
Everything worked fine for me after that