VMware Networking Community
TryllZ
Expert
Expert
Jump to solution

VM Not reaching Internet via Edge ?!

Hi All,

My NSX network setup is as follows, there are no firewall restrictions anywhere.

TryllZ_0-1692393854208.png

Both VMs can ping each other which tells me T1 Gateway is working.

Both Edge nodes can ping firewall interface and internet as well.

TryllZ_0-1692394407588.png

TryllZ_2-1692394057283.png

TryllZ_1-1692394043349.png

Its the VMs that cannot reach the internet, the reply is coming from T0 interface.

TryllZ_3-1692394127207.png

Traceroute results with the following.

TryllZ_1-1692394464844.png

Any thoughts where the issue might be ?

Reply
0 Kudos
44 Replies
DanielKrieger
Enthusiast
Enthusiast
Jump to solution

Were your uplink interfaces not on trunk? Normally it is sufficient to specify only the VLANs you need on the trunk. But I am glad that it works now.

To make the OPNSense untagged the traffic, you have to assign a new interface to the VM and assign the interface directly under assignments. I personally would use a virtual vyos (opensource) router to do the BGP peering. NSX peers to the vyos and the OPNSense also peers to the vyos. The advantage would be you don't have to change your firewall every time you want to test something in BGP and the vyos is much more flexible in BGP configuration.

My Vyos has 2 interfaces, one to the firewall and 1 interface with vlan subinterface for BGP peering with my NSX environment.

----------------------------------------------------------------------
My Blog: https://evoila.com/blog/author/danielkrieger/
Reply
0 Kudos
TryllZ
Expert
Expert
Jump to solution

Were your uplink interfaces not on trunk? Normally it is sufficient to specify only the VLANs you need on the trunk.

The uplink interfaces were Trunks for 2 VLANs only on each Uplink Portgroup, same as yours, 25, and 24 (Uplink 1), and 26, 24 (Uplink 2).

Earlier

TryllZ_0-1692948398823.png

Now

TryllZ_1-1692948686418.png

I had removed VLANs specified on the Trunks and allowed all (0-4094) on both the Uplink Portgroups in Distributed Switch.

Reply
0 Kudos
Leaha00
Contributor
Contributor
Jump to solution

Hey, did you ever get to the bottom of this, I have been trying to setup an NSX lab at home with an OPNsense router and I am having the same issue, I can ping the physical network, including the physical router, but the traceroute stops dead at the OPNsense VLAN GW thats connected to NSX?

I have Physical Router --> OPNSense --> NSX

Thanks

Reply
0 Kudos
TryllZ
Expert
Expert
Jump to solution

Hi,

I'm afraid no, the only way I got this to work was by allowing all VLANs through the Distributed Switch NSX Uplink portgroups (0-4094), which as far as I know is a valid VMware design.

Reply
0 Kudos
DanielKrieger
Enthusiast
Enthusiast
Jump to solution

@Leaha00 

I have the same Setup at a Customer and in my Homelab (fully Nested) and it runs very well. 

My Setup is Nested Edge on virtual ESXi hosts running on pysical ESX Hosts -> vyos Routter -> virtual PFSense Cluster (not Nested, runs on my physical ESX Servers) PPPoE dial IN or if PPPoE Fails for some reasons -> LTE Router - Internet

I use BGP ECMP between Edge - Vyos - PFSense.

You have to setup your routing right and your Outbound NAT on the PFSense/OPNSense. Even if the Networks get lerned from BGP, the Outbound NAT will not work Out of the Box, because your NSX Segment is not a Connected Network at the Firewall.

 

----------------------------------------------------------------------
My Blog: https://evoila.com/blog/author/danielkrieger/
Reply
0 Kudos