WSL2 really slow on Windows 11 - VMWare Fusion

[SteveBall]

I've got a 2019 MBP (32GB RAM) running Monterey (12.0.1) with VMWare Fusion 12.2.3 installed

I've got a Windows 11 VM installed (with a TPM added to Devices), and need to use WSL2.

VM has 8GB Ram, 4 processors allocated.

W11 installed fine, but calling WSL2 is SO slow. On another PC with just Windows installed, my colleagues report it's really quick.

Is anyone else using W11 and WSL2 on a mac host and seeing the same issue?

[SteveBall]

I'm wondering if this might be related to the number of speed issues being reported with 12.2.x

https://communities.vmware.com/t5/VMware-Fusion-Discussions/12-2-0-VMs-really-slow-compared-to-12-1-...

I'm going to try downgrading to 12.1.2 and see if that helps. 

[Technogeezer]

Are you using an encrypted VM to get that TPM device, or the experimental vTPM?

[ColoradoMarmot]

And what's the host config?  If it's a 4 core machine, drop back to 3 or 2 guest CPU's.  (not that hyperthreading/virtual cores don't count).

[Technogeezer]

I did notice that you're 3 updates behind on Monterey updates. Have you considered upgrading to the latest macOS 12.4 and see if the symptoms change?

How does Windows run outside of WSL2? 

Is your virtual machine configured to allow hypervisor applications?

I'm wondering since WSL2 uses Hyper-V components if this is technically considered nested virtualization. If so, I can think of one of two things right off the bat (referring to https://kb.vmware.com/s/article/80785 about limitations of nested virtualization)

• Do you have side channel mitigations disabled for the VM? If so, turn it off and see if the situation improves.
• Does this particular Mac model's CPU support VMCS shadowing? If not, Fusion is going to drop to a software emulation which is going to slow down the VM. There's nothing you can do about that since VMware's software solution is working around a constraint given to it by Apple's hypervisor framework.
  
  According to this article, https://communities.vmware.com/t5/VMware-Fusion-Discussions/Attempting-to-do-nested-virtualization/t... checking the vmware.log for specific entries may tell if VMCS shadowing is available on your Mac if indeed you have allowed nested virtualization in your VM settings. 

[SteveBall]

Host is 2.4ghz 8-core i9 + 8GB Graphics card, so it shouldn't be dropping down the cores. 

[SteveBall]

The VM is encrypted, and I added a device, to create a TPM, which was required to install Windows 11

[bluefirestorm]

For encrypted VM with encrypted disk, make sure that the virtual disk is using preallocated otherwise the virtual disk will have runaway growth. It will be impossible to shrink as punching zeroes using sdelete from the VM would result in non-zeroes being written (as it is suppose to be encrypted). The experimental vTPM with version 12.2.x still has a lot of deficiencies (the encryption password is not known to you, cannot move to a different host, the virtual disk is partially encrypted but it would be pain to remove).

As for the CPU, The ones with vPro as "Yes" would have VMCS Shadowing. In a bizarre twist, the 15"/16" MacBook Pro models from 2016 to 2019, the least and most expensive default configuration from the Apple Store does not have vPro. The middle one (in terms of price of default configuration) are the ones that has vPro (and thus VMCS shadowing). So for a 2019 16" MBP, the 2.3GHz i9 has vPro but the 2.6GHz i7 and 2.4GHz i9 does not.

https://ark.intel.com/content/www/us/en/ark/compare.html?productIds=192987,192990,191045

A possible alternative would be to go back to Catalina and run Fusion 12.1.2 to avoid using the Apple hypervisor framework. The Virtual Machine Monitor (VMM) used in Catalina and earlier macOS is a kext (so probably higher priority as well) and makes direct VT-x calls instead of having additional overhead of Apple hypervisor API at user level privilege.

[SteveBall]

Thanks, @bluefirestorm for the detailed response. 

Unfortunately, for work, I need to have the latest macOS installed.

f it used to be OK on Catalina, do you know if the same issues in macOS Monterey impacting other Virtual Machine software platforms?
Is this something VMWare are like to be able to work around in the future, or is this down to Apple to help resolve?

[bluefirestorm]

Starting with Big Sur, VMware Fusion VMM makes Apple Hypervisor Framework API calls instead of direct Intel VT-x calls. That alone already has some performance penalty although it may not be so obvious.

The VMCS Shadowing is an Intel CPU feature that is useful for nested virtualisation (i.e. running VM within a VM such as WSL2 inside a Windows VM or running docker inside a VM); as it reduces the amount and need for VM entry/exit. VM entry/exit is akin to a process context switch of an application in a regular OS, as it is pausing execution of the VM (VMexit) and control pass back to hypervisor to do something for the VM and then execution of VM resumes (VMentry), and this is where the slowness of VMs generally come from and cost in terms of CPU cycles. With nested VM, there are two hypervisors (one on the host, in this case Fusion VMM on macOS) and the other one inside the VM (Windows Hyper-V). So if WSL2 has to VMexit, chances are the Windows VM itself would also have to transition to the macOS host VMM. This where the VMCS shadowing is trying to prevent in this nested virtualisation scenario.

If this is a work laptop, maybe you should try to replace the 2.4GHz i9 model with a 2.3GHz i9 model that has the vPro. But I don't know how really beneficial VMCS Shadowing would be because if either Hyper-V or Apple Hypervisor framework or both are not using VMCS shadowing or not using it efficiently enough, then it is all for nought. Or another alternative is to run Windows 10/11 as Bootcamp native although this might interrupt or not suit your workflow.

[Technogeezer]

Or consider running a Linux environment as a VM instead of under WSL2. 

[dempson]

---

@SteveBall wrote:

Host is 2.4ghz 8-core i9 + 8GB Graphics card, so it shouldn't be dropping down the cores. 

---

As noted by @bluefirestorm that processor doesn't have vPro or support VMCS shadowing therefore nested virtualisation will perform badly.

I happened to buy the "right" model of the 16-inch MacBook Pro, with the 2.3 GHz 8-core i9 (Core i9-9880H) which does support VCMS shadowing, but I haven't needed that feature in anger yet because I'm still running VMware Fusion 12.1.x on Catalina when I need to use VMs. I don't want to have to deal with the various other annoyances people keep reporting here, which seem to be due to limitations in Apple's hypervisor. Here's hoping they improve it in macOS 13.

I have the luxury of that Mac no longer being my main computer, so it isn't a hassle to leave it running an older macOS version, but the end of security updates later this year will increase the risk of continuing to avoid the Apple hypervisor.

As for your later question about other virtualisation platforms: at the moment it seems that Parallels lets you use either the Apple hypervisor or their own one on Big Sur and Monterey. Third party kernel extensions are already deprecated, and they will be eliminated in a future macOS version, but Apple hasn't yet told us which version. When that happens, Parallels would also be forced to use Apple's hypervisor.