VMware Networking Community
bhards4
Hot Shot
Hot Shot

unable to establish east-west communication using NSX-T

Hi All,

I have setup lab of NSX-T 2.4 where Im trying to communicate East-West communication but after multiple attempt Im unable to establish the communication amount 2 diffrent subnet VM's

My lab environment

vmware workstation 15.5.0

NSX-T 2.4

ESXi 6.7

vCenter 6.7

Created 3 network on Workstaton 15.5.0

vmnet-8 ( For management Network) 192.168.0.x

vmnet-0 (vmnetwork) 192.168.1.x)

vmnet-1 (vmnetwork) 192.168.2.x)

I have created T-1 Gateway and 2 segments

Advertise Route (All Static Routes & All Connected Segments & Service Ports) on T-1 Gateway.

Seg-1 subnet 192.168.1.1

Seg-2 subnet 192.168.2.1

Created 2 VMs on vCenter and allocated IP

vm1 192.168.1.2

vm2 192.168.2.2

Added the segment on the VM network as per the IP address.

Now when I try to ping vm2 ip from vm1 VM, I'm unable to ping but only able to ping the default gateways of both the segments.

Please let me know if I missed somewhere.

-Sachin

Tags (2)
Reply
0 Kudos
23 Replies
vitalsign0
Enthusiast
Enthusiast

Did you specify an Edge Cluster when creating the T1 router? If so, even without using services, traffic is going to try and go through the SR.

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

Well, As per my understanding there is no necessity to deploy edge nodes or cluster on T1 gateway under NSX-T . As by default "DR" Distributed router get installed on each transport node which in my case are ESXi Host which take care of floating distributed services across nodes.

To answer your question : No I have not deployed Edge nodes and cluster under T1 Gateway.

Adding points:

As stated I have 2 segments created using 2 different subnets.

I  have deployed 2 VMs connected with  2 segments having different subnet. Whenever, both the VMs ; hosted on same ESXi host, the east west communication works well and there is no down status identified on Node, Transport zone or TEP on ESXi host. However, I'm unable to find any active TEP under the monitoring section of Node in NSX-T UI.

Now, If i move one VM to other ESXi host, then whole thing breaks down where the communication between both the VMs lost. TEP status shows as down on both the ESXi host, Node and Transport zone status shows as down status.

-Sachin

Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

So the problem is in your TEP network. The tunnels you see in the monitoring section are only present if there are VMs connected to overlay networks. If there is nothing it makes no sense to establish these tunnels, which continuously test connection between TEPs to ensure all is working. That's why when you move the VM you get the errors, because then monitoring begins.

What network have you used for TEP? When you prepared you host TNs what transport network did you use and how did you configure this? Host preparation creates a vmk10 interface used for TEP communication and this is not working between the hosts in your lab.

Reply
0 Kudos
daphnissov
Immortal
Immortal

Your problem is probably the MTU as a common complaint with later versions of Workstation is that they broke the higher MTU abilities. If you cannot ping between TEPs with this command, then this is the issue.

vmkping -S vxlan <TEP> -d -s 1572 -c 10

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

Yes, Im unable to ping the TEP IP's.  Is there any solution to resolve this..

Reply
0 Kudos
daphnissov
Immortal
Immortal

If you can't ping between TEPs then nothing is going to work. I am not aware of a solution of increasing this MTU when using a modern version of Workstation. This is yet another reason that nested, complex labs involving NSX don't usually work too well.

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

hmm, so does that mean VMware workstation has compatibility issue with NSX-T version.

Any further comments.

-Sachin

Reply
0 Kudos
daphnissov
Immortal
Immortal

Any further comments.

Don't use Workstation/Fusion for a nested NSX-T lab. That is all.

Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

But can you ping with a smaller packet size, something less than 1472? MTU might not be the only issue here.

This is why I asked some other questions:

What network have you used for TEP? When you prepared you host TNs what transport network did you use and how did you configure this? Host preparation creates a vmk10 interface used for TEP communication and this is not working between the hosts in your lab.

Regarding MTU it is possible to enable jumbo frames on VMware Workstation:

Enable Jumbo Frames on Windows Host

Reply
0 Kudos
daphnissov
Immortal
Immortal

Regarding MTU it is possible to enable jumbo frames on VMware Workstation:

Enable Jumbo Frames on Windows Host

From some of the other reports I've seen, this functionality appears to be broken. There was a post a while back by Mike Roy acknowledging this somewhere.

Reply
0 Kudos
hindusthan_kash
Contributor
Contributor

Reply
0 Kudos
daphnissov
Immortal
Immortal

This document describes how to build a NSX-T nested lab on vSphere, not one of the desktop products like Workstation.

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

Hi,

Please find the response below

What network have you used for TEP? - Its on the same subnet where the ESXi and the NSX appliance is "192.168.0.x"

When you prepared you host TNs what transport network did you use: I have prepare both the ESXi host using Transport node profile.

Host preparation creates a vmk10 interface : vmk10 interface is showing up on both the ESXi host but im unable to ping the TEP using vmkping on both the ESXi host.

-Sachin

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

Tried enabling Jumbo frame on workstation network but still the same.

Reply
0 Kudos
mauricioamorim
VMware Employee
VMware Employee

But can you ping with a smaller packet size, something less than 1472? MTU might not be the only issue here.

Send us some screenshots of your ESXi host network configuration, TN profile, etc.

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

Hi Mauricioamorim,

Yes, Im able to ping the TEP with lower  1470-75 as well as with jumbo frame 9000 but still unable to communicate between TEP-TEP amount 2 ESXi host.

Please find some of the screenshots of Traceflow and Node down status of NSX-T .

-Sachin

Reply
0 Kudos
daphnissov
Immortal
Immortal

Yes, Im able to ping the TEP with lower  1470-75 as well as with jumbo frame 9000 but still unable to communicate between TEP-TEP amount 2 ESXi host.

So you're testing by pinging the TEP on host-A...from host A itself? That proves nothing and of course you can do that. You're still not proving anything with your traceflow screenshots. If you can't ping from host-A to host-B between the TEPs, even at a smaller MTU size, then you have a fundamental networking connectivity problem that transcends NSX-T.

Reply
0 Kudos
bhards4
Hot Shot
Hot Shot

@daphnissov Im able to do vmkping of TEP between both the ESXi host without any issue , no matter if the mtu is smaller or jumbo frame. So nothing fundamentally wrong anywhere.

As stated, the issue is the same, where im not able to perform East-West connectivity of vms which are on diffrent subnet hosting on individual ESXi host.

Let me know if have have any solution for this.

-Sachin

Reply
0 Kudos
daphnissov
Immortal
Immortal

Ok, now I'm confused because previously you said you cannot ping between TEPs. Now suddenly you can? How about tell us what the TEPs are between your hosts and show the console output of the command I provided you earlier listing the destination TEP of the other host.

Reply
0 Kudos