VMware Networking Community
APJ_vm
Enthusiast
Enthusiast

Connecting VM to segment causes segment to go down

Hi 

First very new to NSX-T, so excuse me if I use the incorrect terminology. I have roughly followed this setup -> Tier-0 Gateway - The IT Hollow

The issue I am seeing is, if I have no VM's connected to any of the segments, I am able to ping the segment gateways. The moment I connect a VM to any segment all communications to the segment gateways goes down. When I look at the Transport Zones my overlay TZ shows degraded. The VM connection has cause the edge and the host on which the VM resides to go down. I have a single static route on my T0 of 0.0.0.0/0 with the next hop being my physical switch. Any idea what I have done wrong or might be missing.

Running NSX-T Version 3.1.0.0.0.17107209

NSX-TZ-Status.png

 

NSX-Top.png

Reply
0 Kudos
7 Replies
p0wertje
Hot Shot
Hot Shot

Hi,

 

Are you using physical switches ? If so, what mtu is configured on them ?

 

 

Cheers,
p0wertje | VCIX6-NV | JNCIS-ENT | vExpert
Please kudo helpful posts and mark the thread as solved if solved
Reply
0 Kudos
APJ_vm
Enthusiast
Enthusiast

The physical switch is set for 9000. The DVS is set to 1600.  From the ESXi hosts I have confirmed that I can ping using a 1572  packet size. 

Reply
0 Kudos
p0wertje
Hot Shot
Hot Shot

When you click the "down" does it give you any extra information ?
You could go look on the esx host and use 'nsx-cli' command

get host-switch <switch-name> tunnel
get host-switch <switch-name> tunnels <x.x.x.x> <y.y.y.y>
get logical-switches
get logical-switch <logical-switch id> vtep-table

Might give you more info on why the tunnel is down.

Cheers,
p0wertje | VCIX6-NV | JNCIS-ENT | vExpert
Please kudo helpful posts and mark the thread as solved if solved
Tags (1)
Reply
0 Kudos
APJ_vm
Enthusiast
Enthusiast

There is no information related to why things are in a down state. 

Below is the output of some of the commands you mentioned. 

host1> get host-switch Homelab-DVS tunnel
Sun May 02 2021 UTC 21:21:15.101
      Local IP            Remote IP       Local State  Remote State
   192.168.202.1       192.168.202.253        down         down
   192.168.202.1       192.168.202.252        down         down
host1> get host-switch Homelab-DVS tunnel 192.168.202.1 192.168.202.253
Sun May 02 2021 UTC 21:22:38.262
    Local State      :        down
    Remote State     :        down
     Local Diag      :   No Diagnostic
    Remote Diag      :   No Diagnostic
       minRx         :        1000
       minTx         :        1000
     Local Disc      :     0x7cdcdc5a
    Remote Disc      :        0x0
    Tx Interval      :        1000
    Rx Interval      :        1000
        mult         :         3
host1> get logical-switches
Sun May 02 2021 UTC 21:24:49.646
                  Logical Switches Summary
------------------------------------------------------------

                    Overlay Kernel Entry
============================================================
  VNI                    DVS name                 VIF num
 73734                 Homelab-DVS                   2
 73738                 Homelab-DVS                   1
 73735                 Homelab-DVS                   1
 73736                 Homelab-DVS                   1
 73737                 Homelab-DVS                   1

                     Overlay LCP Entry
============================================================
  VNI              Logical Switch UUID              Name
 73734     25af8478-36c1-4f0e-9b69-236d4fa9d2f5   SEG-1
 73738     e7113bad-dfb4-4f78-bd84-64fc037ca8a1   SEG-4
 73736     50f8e019-2188-41bb-b7ee-57eec010a63d   SEG-2
 73737     ae4e8c2b-4c76-43f1-bb2b-2d6fa3d9a719   SEG-3
 73735     aa54ddaf-6536-46fc-8e92-65b1c7b7bf78   transit-bp-d9a418b3-317b-4108-b11b-67d123d9a027

                     VLAN Backed Entry
============================================================
          Logical Switch UUID            VLAN ID

 

host1> get logical-switch 25af8478-36c1-4f0e-9b69-236d4fa9d2f5 vtep-table
Sun May 02 2021 UTC 21:25:52.246
                                   Logical Switch VTEP Table
-----------------------------------------------------------------------------------------------

                                       Host Kernel Entry
===============================================================================================
 Label        VTEP IP           Segment ID     Is MTEP       VTEP MAC       BFD count

                                       LCP Remote Entry
===============================================================================================
 Label        VTEP IP           Segment ID          VTEP MAC                  DEVICE NAME

                                        LCP Local Entry
===============================================================================================
 Label        VTEP IP           Segment ID          VTEP MAC                  DEVICE NAME
 114695    192.168.202.1      192.168.202.0    00:50:56:61:17:7b                 None

host1> get logical-switch aa54ddaf-6536-46fc-8e92-65b1c7b7bf78 vtep-table
Sun May 02 2021 UTC 21:26:09.253
                                   Logical Switch VTEP Table
-----------------------------------------------------------------------------------------------

                                       Host Kernel Entry
===============================================================================================
 Label        VTEP IP           Segment ID     Is MTEP       VTEP MAC       BFD count
 114691   192.168.202.252     192.168.202.0       0     00:50:56:95:00:1a      0

                                       LCP Remote Entry
===============================================================================================
 Label        VTEP IP           Segment ID          VTEP MAC                  DEVICE NAME
 114691   192.168.202.252     192.168.202.0    00:50:56:95:00:1a                fp-eth1

                                        LCP Local Entry
===============================================================================================
 Label        VTEP IP           Segment ID          VTEP MAC                  DEVICE NAME

 

I have a single edge, with fp-eth0 and fp-eth1 connected to a trunked DVS port group. eth0 is connected to a DVS port group with no VLAN. My edge and my hosts are using the same uplink profile, which has a Teaming policy of Load Balance Source. It does not make sense to me that I am able to ping segment GW when no VM is connecte, then the moment it gets connected things go south. 

 

Reply
0 Kudos
p0wertje
Hot Shot
Hot Shot

- does adding a vm on host1 and a vm on host2 and ping between them (in the same segment) work ?
- do you have a VM based edge ? If so, are you using the same ip range for the teps ? if so, you need to create a nsx-t vlan backed segment

p0wertje_0-1620028264268.png

 

Cheers,
p0wertje | VCIX6-NV | JNCIS-ENT | vExpert
Please kudo helpful posts and mark the thread as solved if solved
Reply
0 Kudos
zeroboy
Enthusiast
Enthusiast

How did this one end up? Sitting in front of the same problem, but using Nested ESXi using non-nested NSX-T segments.

The moment I connect a VM to the segment, it goes down. TEP are pingable though.

Reply
0 Kudos
APJ_vm
Enthusiast
Enthusiast

I could never get to the bottom of that issue and ended up destroying my NSX-T lab. I have since rebuilt it using the details in this blog 
NSX-T 3.1 Home lab - Setup and Fabric configuration | rudimartinsen.com

Everything worked fine for me after that

Tags (1)
Reply
0 Kudos