Trying to use an additional subnet / vlan

[amirzadeh]

I'm running on ESXi 7, vSphere 7 (no NSX, etc). 

Setup:

pfSense 

+ switch (tp-link switches, vlan aware ... I have tagged all ports with vlan 107).

+ esx host (192.168.1.7)

My base subnet is 192.168.1.0/24, everything there is working well and is connected. I'm testing/playing with Kubernetes and with that I would like to be able to use another subnet as my public ips for the services.  I went with 192.168.107.0/24.

I created a base ubuntu vm, (put it on the std-0-kub-107 network) statically assigned 192.168.107.3, gateway of 192.168.107.3 ... but it can't reach anything and nothing can reach it.

From the ESX host I can ping the 107 interface on pfsense:

[root@esxi7:~] ping 192.168.107.1
PING 192.168.107.1 (192.168.107.1): 56 data bytes
64 bytes from 192.168.107.1: icmp_seq=0 ttl=64 time=0.466 ms
64 bytes from 192.168.107.1: icmp_seq=1 ttl=64 time=0.414 ms

--- 192.168.107.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.414/0.440/0.466 ms

I created a port group and assigned it to the VLAN .

[root@esxi7:~] esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 6400 32 128 1700 vmnic0,vmnic3,vmnic2,vmnic1

PortGroup Name VLAN ID Used Ports Uplinks
std-0-kub-107 107 1 vmnic0,vmnic3,vmnic2,vmnic1
std-0-vlan-0000 4095 18 vmnic0,vmnic3,vmnic2,vmnic1

What step am I missing?

[maksym007]

Is it your nested environment, right? 

Check here if you really configured as here
https://mattadam.com/2021/07/21/virtual-router-in-your-home-lab-pfsense/#Adding_networks_to_pfsense_...

[amirzadeh]

@maksym007I think there was a misunderstanding.  Allow me to clarify - my pfsense is outside of ESX and the router/gateway of the network with various interfaces on it.  192.168.1.0/24 (main LAN on VLAN 1) as well as the 192.168.107.0/24 (VLAN 107).  I can ping the 192.168.107.1 (pfsense interface) from any machine on the 192.168.1.0 network however when I deploy a VM on ESX on the 192.168.107.0 network I cannot ping anything.

[a_p_ · Accepted solution]

I'm not sure whether I fully understand your network setup.
Anyway, in case that the physical switch port to which the ESXi host is connected to access VLAN 107 is an untagged port (or its defailt VLAN is 107), you must not tag the virtual port group with that VLAN ID too.

André

[amirzadeh]

That was it, I was tagging both the port and the port group.  Thank you.