VMware Networking Community
MRoushdy
Hot Shot
Hot Shot

NSX-T - Cross-site requirements

Hello.

I\m trying to simulate NSX-T multi-site in my home lab, the remote host was configured for NSX successfully, but, traffic is not passing over GENEVE tunnel.

SO, I\m simulating SITE01, and SITE02, the setup is as following:

SITE01

*********

1 NSX-T Manager

2 Edge nodes (ESMP)

2 ESXi transport nodes

This site works very fine for intra-overlay, I mean that VMs on overlay communicate successfully.

SITE 2

*********

1 ESXi transport node, with 2 vms on it, and both run on different segments, and can ping each other successfully.

Networking equipment:

**************************

Site 1: Cisco 3750 switch >>> connected to Cisco 1841 router (the gateway to site 2)

Site 2: Cisco 1841, connected to site one, and to the local ESXi host on that site.

Problem: inter-site overlay traffic is not passing.  Normal TEP ping works, but this command isn't working across sites

[root@esxi05:~] vmkping -S vxlan 10.150.10.11 -d -s 1400 -c 10

PING 10.150.10.11 (10.150.10.11): 1400 data bytes

pastedImage_0.png

So, what'\s the proper configuration for NSX-T inter-site communication please?

vEXPERT - VCAP-DCV - Blog: arabitnetwork.com | YouTube: youtube.com/c/MohamedRoushdy
Reply
0 Kudos
5 Replies
serbl
Enthusiast
Enthusiast

Hi,

Is the MTU set to at least 1600 on the link between the sites?

Best regards, Rutger
Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

Nothing special is needed for multi-site NSX-T to work. It is not aware if it is two different sites or only a layer 3 boundary. As long as you have L3 connectivity among TEPs with adequate MTU it just works. Make sure routing between different TEP networks is OK and that they have default gateway assigned. Also check MTU end-to-end.

Reply
0 Kudos
MRoushdy
Hot Shot
Hot Shot

Routing is defined fine, I use RIP in my lab, but maybe it's all about the default gateway, and MTU. In my main site, hosts could send N/S traffic via the edge node, which has TEP on a different VLAN used for host TEP, and this means that routing here is fine. How can I set a default gateway please? I\m using DHCP pool to assing IPs to TEPs,but when I check the interfaces from ESXcCLI, ono of the TEPs has a gateway defined, is this normal? THe gateway is "0.0.0.0".

vEXPERT - VCAP-DCV - Blog: arabitnetwork.com | YouTube: youtube.com/c/MohamedRoushdy
Reply
0 Kudos
MRoushdy
Hot Shot
Hot Shot

MTU is defined every where, but I will check that again.

vEXPERT - VCAP-DCV - Blog: arabitnetwork.com | YouTube: youtube.com/c/MohamedRoushdy
Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

ESXi has a different stack fot TEP traffic called vxlan (although it uses GENEVE in NSX-T), which needs its own default gateway if you have TEPs in different subnets. You can check this in vcenter on each host in Configure > Networking > TCP/IP Configuration. There you can see if a default gateway is configured on this network stack. Since you're using DHCP for configuring TEPs, default gateway has to be defined on the DHCP server.

Reply
0 Kudos