VM Not reaching Internet via Edge ?!

[TryllZ]

Hi All,

My NSX network setup is as follows, there are no firewall restrictions anywhere.

Both VMs can ping each other which tells me T1 Gateway is working.

Both Edge nodes can ping firewall interface and internet as well.

Its the VMs that cannot reach the internet, the reply is coming from T0 interface.

Traceroute results with the following.

Any thoughts where the issue might be ?

[DanielKrieger]

Were your uplink interfaces not on trunk? Normally it is sufficient to specify only the VLANs you need on the trunk. But I am glad that it works now.

To make the OPNSense untagged the traffic, you have to assign a new interface to the VM and assign the interface directly under assignments. I personally would use a virtual vyos (opensource) router to do the BGP peering. NSX peers to the vyos and the OPNSense also peers to the vyos. The advantage would be you don't have to change your firewall every time you want to test something in BGP and the vyos is much more flexible in BGP configuration.

My Vyos has 2 interfaces, one to the firewall and 1 interface with vlan subinterface for BGP peering with my NSX environment.

[TryllZ]

Were your uplink interfaces not on trunk? Normally it is sufficient to specify only the VLANs you need on the trunk.

The uplink interfaces were Trunks for 2 VLANs only on each Uplink Portgroup, same as yours, 25, and 24 (Uplink 1), and 26, 24 (Uplink 2).

Earlier

Now

I had removed VLANs specified on the Trunks and allowed all (0-4094) on both the Uplink Portgroups in Distributed Switch.

[Leaha00]

Hey, did you ever get to the bottom of this, I have been trying to setup an NSX lab at home with an OPNsense router and I am having the same issue, I can ping the physical network, including the physical router, but the traceroute stops dead at the OPNsense VLAN GW thats connected to NSX?

I have Physical Router --> OPNSense --> NSX

Thanks

[TryllZ]

Hi,

I'm afraid no, the only way I got this to work was by allowing all VLANs through the Distributed Switch NSX Uplink portgroups (0-4094), which as far as I know is a valid VMware design.

[DanielKrieger]

@Leaha00 

I have the same Setup at a Customer and in my Homelab (fully Nested) and it runs very well. 

My Setup is Nested Edge on virtual ESXi hosts running on pysical ESX Hosts -> vyos Routter -> virtual PFSense Cluster (not Nested, runs on my physical ESX Servers) PPPoE dial IN or if PPPoE Fails for some reasons -> LTE Router - Internet

I use BGP ECMP between Edge - Vyos - PFSense.

You have to setup your routing right and your Outbound NAT on the PFSense/OPNSense. Even if the Networks get lerned from BGP, the Outbound NAT will not work Out of the Box, because your NSX Segment is not a Connected Network at the Firewall.