Fusion 12.2.0 networking for guest machine issues

[DVE2000]

I just installed Fusion 12.2.0, and then noticed my email stopped working. I've had a VM machine running Win10 and SmarterMail for years on my various Macs. Last time we had any network issues was with version 8.sumthin and port forwarding.

With this version, none of my guest machines had internet access. I tried changing to different vmnet's, bridging mode etc, run the cli start and stop commands and finally the guest machine for my mail server said it had a network connection. I got all guest machine's networks working again.  Anyway, I shut down all machines, shut down the Fusion client, and then ran the command line to start the mail server machine up again. (I have scripts to start the mail server up with a LaunchDaemon plist, and make sure it's shutdown when my Mac is shutdown.) And the network had stopped working again. And then things like this:

[Mac] > sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop

Stopped all configured services on all networks

[Mac] > sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start

Enabled hostonly virtual adapter on vmnet1

Failed to start DHCP service on vmnet1

Failed to start NAT service on vmnet3

Enabled hostonly virtual adapter on vmnet3

Failed to start DHCP service on vmnet3

Failed to start NAT service on vmnet8

Enabled hostonly virtual adapter on vmnet8

Failed to start DHCP service on vmnet8

Failed to start some/all services

I messed around for another half an hour, and then finally just downgraded to 12.1.2 again. Everything is just fine with all the networking. The strange thing is that the release notes for 12.2.0 says that some networking issues have been fixed! It definitely was a headache for me, and I'm happily back at the previous rev. I hope things are REALLY fixed with the next release.

[DVE2000]

Oh, and I am on maOS Big Sur 11.6 with my Mac Mini.

[AJBDTI]

I can confirm the same issue here on macOS 11.6 and Fusion 12.2. The update to 12.2 killed all the networking options for all of our VM's.

[OS2EPM]

I've had luck reducing MTU (to 1200 in this case) on the VMs. I'm able to get updates again, at least, over WI-FI, in my VMs.

[ivivanov]

Hello,

A quick question just to confirm: was VMware Fusion UI running while you have tried to start and stop networking services with vmnet-cli? I was able to reproduce the issue while the VMware Fusion UI was stopped, but if it was up and running then the services were started successfully. So I would like to confirm if you are seeing the same issue, or it is something different?

[OS2EPM]

Apologies - I read your symptoms as the GUESTS were having trouble with networking, not the host.

I had done no further troubleshooting on the host.

Strangely enough, I have had to re-log-in to a Hotmail account, but I believe that a coincidence.

I was able to fix the GUEST networking issues by reducing MTU. I have not (yet) experienced trouble on the host.

[DVE2000]

That's correct, I couldn't start the networking with the cli if the UI was stopped. However, even if the UI was running and I could use the cli to stop and restart the networks, the guest's network connections showed as not connected to the internet. I have no real idea how I got it working the first time. I think I may have copied my custom vmnet3 file back over time machine and stopped and restarted network services. I think they started getting connections when I disconnected and reconnected the network adapter in the guest machine settings. I know I'd also tried bridging, share with my mac, etc to try get *something* to work. Most of the time, it didn't work/help.

[DVE2000]

Oh, and BTW, I'm using a wired ethernet connection to my mac. I do have wifi connected, but its priority is set lower than the wired connection.

[ivivanov]

> Oh, and BTW, I'm using a wired ethernet connection to my mac.

That was what I was about to ask. So, to summarise, the network is not working for you in neither of the following configurations:

* bridged mode (using Ethernet)

* NAT/Shared mode - WITH networking services running (using Ethernet)

What is your guest IP configuration? Is it using DHCP? If yes, is a valid IP address successfully assigned to the VM, or the IP address defaults to 169.254.x.x (in both modes)? If it is manually assigned - can you double-check the IP configuration is valid (e. g. matches the expected physical configuration in bridged mode, has the .2 IP address as a DNS server and default gateway in NAT mode)? Is there an entry in the /var/db/vmware/vmnet-dhcpd-vmnet8.leases (for NAT mode when services are running) for the specific VM (i. e. the MAC address)? Just to confirm - when powering on a VM you do NOT see a popup message saying Fusion could not start a virtual network adapter, and the vNIC state is "connected"? The default configuration (Shared mode + DHCP in the guest (and networking services running on the host)) should be fairly reliable.

[DVE2000]

I just checked, and all 3 VMs that I'm mostly using (and checked) have fixed ip addresses set. I'm not using DHCP. I did this way back when because I wanted to be able to reliably RDP to the guest machines.

The reason I'm using vmnet3 is that I'm forwarding some ports. I discovered years ago that vmnet8 settings are sometimes overwritten (like on a Fusion s/w upgrade). But only one VM, the mail server, is using vmnet3. The subnet is 192.168.17.0, and the gateway is 192.168.17.1. The DNS server is set to my real router, which is on my 192.168.1.0 subnet. Another VM is set to "share with my mac" (vmnet8)., but it's also set to a fixed ip address on the 192.168.16.0 subnet. Both gateway and DNS servers are 192.168.16.1. Similar to the last for the final VM.

I didn't check to see if vmnet8 has had the default subnet changed, but if it had, I would expect to see the issues that I did on two of the machines. The one using vmnet3 had issues also, but I could see the subnet in the Fusion preferences, and that was still at 17. I wouldn't have expected to see issues with that VM.

I totally get that you'd like me to upgrade again to help debug and root cause this, but I'm kind of reluctant seeing that I have my mail server working again. I had a bad few hours last night. 😛 

A data point that is really interesting though is that my mail server didn't have access to the internet. HOWEVER, I was able to RDP to it just fine! And when I disabled the network adapter in the RDP session, the session was immediately terminated. Which I would expect. I could use the GUI to turn the network adapter on again, but the guest vm would still say it had no connection to the internet. I don't think it could even ping my Mac, but I think RDP started working again. I also used 'nc -zv <my server domain> <port>' in terminal on my Mac to check the mail server ports, and the ports were just fine, even though the machine supposedly wasn't connected to the internet!

I never saw any messages saying that the virtual network adapter(s) had any issues. The guest operating systems knew if I stopped the network service with the cli, and knew when I started it up again. So it was the equivalent of a network port having a wire to a switch or router, but the switch or router not quite working properly

[ivivanov]

> I totally get that you'd like me to upgrade again to help debug and root cause this

Sorry, I somehow ignored the fact that you have downgraded back to 12.1.2 already, I was just trying to help 😊.

However I think your description is enough to identify the root cause - you said you are using a static IP configuration and the gateway is 192.168.17.1 (for vmnet3), and 192.168.16.1 (for vmnet8). There is a "change in the behaviour" in Fusion 12.2 compared to Fusion 12.1 - the default gateway should be set to .2 address, but not to 1 (.1 is used for a host IP address attached to the same vmnet). Actually this "restores" the behaviour we had in Fusion 11.x, and it was changed in Fusion 12, caused by a limitation in the Apple network virtualization API that we are using on Big Sur and above. This is also confirmed by the RDP experience that you have described. First, it shows the virtual NIC is configured correctly and there is traffic sent through it to VM, and second, since you are using port-forwarding, there is a direct host-guest communication, without the need of the gateway to be used inside the guest OS. However if you do try to reach to an IP address outside of the current subnet, then the traffic should be routed through the gateway and it is wrong (it should be .2 instead of .1). There is no chance for a static IP configuration to work in bridged mode (without updating the configuration to match the physical network) - again the VM cannot reach its default gateway, so no traffic in and out.

That said I am leaving it up to you when/if you would like to give Fusion 12.2 another chance 😜. Thanks for your feedback.

[DVE2000]

I’m happy to upgrade again and fix the settings. The only thing I’m wondering about now is how the virtual network daemon works when the gui isn’t running because of the stop/start issues. Like I said, the mail server runs headless, because I have it started when by a script that runs when my Mac starts up. That script is in an infinite loop, but sleeps for 24 hours a day. If it catches any signals, it stops the mail server so that things shut down gracefully. But I sometimes fire up the gui to run the other machines. When I shut the gui down again, the mail server is suspended (sometimes — depends if I just start the gui, or run another vm from the taskbar menu), but I can start it up from a terminal session. Is the network daemon going to work just fine in this case?

Another question — is the signal detection script even needed these days? If a machine is running headless, will it just suspend if the Mac is shutdown?

[ThomasMax]

Can confirm that rolling back to 12.1.2 solved my sudden networking issues as of 12.2.0 on Big Sur 11.6, where guest VMs connected via private virtual network with NAT couldn't connect to the Internet anymore and couldn't ssh to the VMs from hosts on the physical network (via port forwarding).

[ivivanov]

Ohh, I completely forgot about the "background" mode without an UI. In this case you'd better not upgrade to 12.2 😊.

Seriously, I was not aware of this mode of operation. I am not sure if we support it and test it. However given the network services do not start if the UI is not running is a blocker for NAT mode. For bridged - it should be OK since those services are not needed. A quick question - are you able to start those services with vmnet-cli without UI with Fusion 12.1.2? I remember there were some security-related improvements in 12.0, however no network services were needed for 12.0/12.1.x NAT mode, except for the port-forwarding).

For the "signal-detection" script - I would expect it is still needed. Fusion suspends all running VMs on quit, however I kind of assume it is a feature of the UI, but not the running VM itself (not positive about that though).

[ivivanov]

OK, I just tried locally with Fusion 12.2. Indeed the network services could not be started with "sudo vmnet-cli --start".
HOWEVER "sudo vmrun start Ubuntu.vmx" started the services successfully, the VM was started and the networking was working both in NAT and bridged mode.

[DVE2000]

Thanks! That’s good enough for me! I’ll upgrade tomorrow and report back. 

[DVE2000]

I'm happy to report that everything is working sweetly with 12.2.0. Thanks for all the info and help!

I changed the gateway and dns servers on the vmnet8 machines and they're all good. Changed the gateway for the mail servers, and that's fine as well. And as long as a machine is running, even if the gui is not, it's possible to stop and start the networks from the command line

[ivivanov]

I am glad Fusion 12.2 is working for you. Thanks for sharing.