amirzadeh
Contributor
Contributor

ESXi 7 - VM can ping gateway on VLAN interface, but gw cannot ping device

I have multiple VLANs in my homelab.

VLAN 1 (default)
VLAN 10 (guest) - portgroup: std-0-guest-10
VLAN 66 (IOT) - portgroup: std-0-iot-66
...

Gateway is a PfSense+ on a Netgate 3100 -- everything works if I use an external DNS for the network (via DHCP) if I use the internal DNS that I would like to use, nothing resolves.

I'm running an Ubuntu VM that runs DNS and I need it to have available on all VLANs.  The device has multiple vEth, on each network.  The Guest VLAN works, gateway can ping server, server can ping gateway, DNS works.  Device on IOT VLAN cannot get to DNS.

From server:

# ping -I ens224 192.168.66.1
PING 192.168.66.1 (192.168.66.1) from 192.168.66.80 ens224: 56(84) bytes of data.
64 bytes from 192.168.66.1: icmp_seq=1 ttl=64 time=0.947 ms
64 bytes from 192.168.66.1: icmp_seq=2 ttl=64 time=0.406 ms

From gateway:

PING 192.168.66.3 (192.168.66.3) from 192.168.66.1: 56 data bytes

--- 192.168.66.3 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss
# ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens160: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:50:56:8f:7c:64 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.3/24 brd 192.168.1.255 scope global ens160
valid_lft forever preferred_lft forever
3: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:60:87:79 brd ff:ff:ff:ff:ff:ff
inet 192.168.107.3/24 brd 192.168.107.255 scope global ens192
valid_lft forever preferred_lft forever
inet 192.168.107.130/24 brd 192.168.107.255 scope global secondary noprefixroute ens192
valid_lft forever preferred_lft forever
4: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:50:56:bd:5b:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.66.3/24 brd 192.168.66.255 scope global ens224
valid_lft forever preferred_lft forever
inet 192.168.66.80/24 brd 192.168.66.255 scope global secondary noprefixroute ens224
valid_lft forever preferred_lft forever
5: ens256: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:50:56:bd:ca:34 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.3/24 brd 192.168.10.255 scope global ens256
valid_lft forever preferred_lft forever
inet 192.168.10.10/24 brd 192.168.10.255 scope global secondary noprefixroute ens256
valid_lft forever preferred_lft forever
6: v10@ens256: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:50:56:bd:ca:34 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.3/24 brd 192.168.10.255 scope global v10
valid_lft forever preferred_lft forever
7: v66@ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:50:56:bd:5b:45 brd ff:ff:ff:ff:ff:ff
inet 192.168.66.3/24 brd 192.168.66.255 scope global v66
valid_lft forever preferred_lft forever
8: v107@ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:60:87:79 brd ff:ff:ff:ff:ff:ff
inet 192.168.107.3/24 brd 192.168.107.255 scope global v107
valid_lft forever preferred_lft forever
# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 ens160
0.0.0.0 192.168.107.1 0.0.0.0 UG 0 0 0 ens192
0.0.0.0 192.168.66.1 0.0.0.0 UG 0 0 0 ens224
0.0.0.0 192.168.10.1 0.0.0.0 UG 0 0 0 ens256
0.0.0.0 0.0.0.0 0.0.0.0 U 0 0 0 v107
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 ens160
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 ens256
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 v10
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 v66
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 ens224
192.168.107.0 0.0.0.0 255.255.255.0 U 0 0 0 v107
192.168.107.0 0.0.0.0 255.255.255.0 U 0 0 0 ens192

I used `vim-cmd vmsvc/device.getdevices 27` to see if I can see any differences attaching it just for reference.

Reply
0 Kudos
lukasrueckerl
Enthusiast
Enthusiast

Hi, 

any specific reason for the Ubuntu DNS-Server to need multiple NICs on each VLAN?

Having multiple default gateways in your VMs routing table most likely leads to these problems. To check, you could disable all other NICs except the IoT VLAN one and see if the IoT-Machine can ping the DNS. 

If you're using a firewall I would go with a single-NIC DNS-Server, establish Routing and proper Rules and then use the PFSense DNS Forwarder described here: Services — DNS Forwarder | pfSense Documentation (netgate.com)

 

Regards,
Lukas

-------------------------------------------
VCIX-NV + VCIX-DCV 2023
Please mark resolved issues as such. Kudos are appreciated.

View solution in original post

amirzadeh
Contributor
Contributor

I actually tried the forwarder before and the DHCP process would not complete for some reason - I moved away from it and setup the extra NICs.  That started to work for some interfaces but obviously I ran into a problem after a few of them.  I just removed all of the NICs and changed the DNS in DHCP to the local interface address and it started to work so I now I'm not sure what the original problem was.  Thank you for the recommendation to try it again, so much simpler.

Reply
0 Kudos