[WSL2][NestedVirt][Cores > 8] VM does not Start when VM is configured to have more than 8 cores.

[martopad]

Hello

My Windows 10 guest VM does not start when I configure it to have more than eight logical processors (greater than 1x8). This is with nested virtualization option enabled. Is this a bug/limitation with workstation pro 16?

Previous OK case

- Configure VM only to have eight or fewer logical processors (I upgraded my PC so that I can configure it to have more than eight logical processors)

OR

- Configure VM to have more than eight logical processors but with nested virtualization disabled (But I need nested virtualization, WSL2 needs it)

OR

- Disable Hyper-V in guest (Same as above, I need virtualization/WSL2)

Additional context

- I am running VMWare workstation pro 16(updated to the latest) in a Windows 10 host (same with Windows 11 host)

- Guest: Windows 10 Enterprise 1909

- 5950x ryzen processor. (previous was 5600x ryzen). Same generation and architecture.

- I have installed VMware tools on the guest machine.

- NOK cases: Processor configs I tried: 1x9, 1x12, 1x24 (set in GUI)

Error message:

- No error message. It just crashes 😞

EDIT:

Attached are the following:

1) Processor Config GUI.PNG - Screenshot of how the processor is configured in GUI

2) VM not starting.PNG - Screenshot notifying that a fault has occurred

3) VM023315-Work.vmx[.txt] - vmx file showing my whole configuration(added .txt at the end so it gets uploaded)

4) vmware.log - log showing what happens when I start VM.

5) Guest OS Build.PNG- Screenshot showing guest OS build

[continuum]

You give no details about your need for nested virtualisation - so shall we assume you want to run a XP VM with Quemu ?

[martopad]

I am running WSL2 inside my guest machine, it requires virtualization platform enabled.

[martopad]

Edited original post to add attachment--screenshots and logs.

[continuum]

Put WSL2 in the headline for better chances.

[bluefirestorm]

What is the edition of the Windows OS in the VM?

I am able to boot up a Windows 11 Enterprise Edition VM (from the MSDN VM downloads, which has WSL2 installed) with vCPUs > 8 without crashing.

You could try editing to use 12 vCPUs from the GUI instead of manually editing the vmx to set it to 9.

[martopad]

Changed title. Thanks for the suggestion

[martopad]

Guest OS build details: Windows 10 Enterprise 1909. I tried setting it to 1x12 cores and no luck. I also tried 1x16, and 1x24--no luck also. (Edited original post to have this info.)

Interesting that yours doesn't crash. I thought this was specific to Workstation pro, since I can boot the same OS build in hyper-v but with more cores and nested virtualization.

[bluefirestorm]

From the log, there was a triple fault.

2022-06-26T19:39:54.231Z In(05) vcpu-0 Triple fault.
2022-06-26T19:39:54.231Z In(05) vcpu-0 MsgHint: msg.monitorEvent.tripleFault
2022-06-26T19:39:54.231Z In(05)+ vcpu-0 A fault has occurred causing a virtual CPU to enter the shutdown state. If this fault had occurred outside of a virtual machine, it would have caused the physical machine to restart. The shutdown state can be reached by incorrectly configuring the virtual machine, a bug in the guest operating system, or a problem in VMware Workstation.---------------------------------------

You could try if MSDN VM, to eliminate whether it is your specific VM build that has the problem.
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/

But I suspect it might have to do with the CPU. I don't have access to a system with an AMD Zen CPU. From what basic understanding I have about the AMD Zen CPUs is it uses the chiplet design. Once you configure beyond the number of cores per chiplet (in this case 8 for the AMD Ryzen 9 5950X), the crossing over to use a core/thread on another chiplet causes the triple fault (since each chiplet has its own L3 cache and probably its own memory controller as well).

So as a possible workaround to have > 8 vCPUs with WSL2 enabled, you could assign CPU thread affinity to the VM to avoid the VM execution being scheduled outside of one chiplet. Shut down the VM and add the following lines to the vmx configuration file. Assuming hyperthreading is enabled, Processor0 to Processor15 would be in one 8 core chiplet.

Processor1.use = "TRUE"
Processor2.use = "TRUE"
Processor3.use = "TRUE"
Processor0.use = "TRUE"
Processor4.use = "TRUE"
Processor5.use = "TRUE"
Processor6.use = "TRUE"
Processor7.use = "TRUE"
Processor8.use = "TRUE"
Processor9.use = "TRUE"
Processor10.use = "TRUE"
Processor11.use = "TRUE"
Processor12.use = "TRUE"
Processor13.use = "TRUE"
Processor14.use = "TRUE"
Processor15.use = "TRUE"

If this workaround does not work, I don't have any other idea.

It is also possible it is a specific combination of Windows 10/11 Hyper-V/WSL2 enabled VM with AMD Zen CPU that causes this triple fault. To eliminate this angle, you could try any of the following (without any workarounds)

Power up a Windows 10/11 VM without WSL2/Hyper-V enabled but with vCPU > 8 but with virtualised AMD-V enabled. You could try something different like Docker or install VMware Player inside this VM.
Create a non-Windows VM (such as ESXi or Linux) with virtualised AMD-V enabled with vCPU > 8

If either non-WSL2 or non-Windows VM or both scenarios still crashes with triple fault with vCPU > 8, it probably reinforces more the chiplet crossover being a problem.

[martopad]

Thank you for the suggestions. It took some time to test but I migrated from Windows 10 host to Windows 11. Unfortunately, the behavior is the same.

1) Trying a different VM installation. I haven't tried this yet.

2) Setting CPU affinity doesn't work. Set cores to 1x16 and added your config in my vmx file. Same issue.

Now for the interesting part.

I followed your next suggestions and it all boils down to enabling Hyper-V in guest. Any VMWare workstation pro settings I used, worked as long as Hyper-V inside guest is DISABLED. (Cores > 8 with AMD-V/RVI enabled)

Nested Virtualization inside the guest enables the other half of my use-cases in this VM though. It would be great to have hyper-V (and in turn WSL2) inside my guest Machine--WSL2 makes my Linux-related tasks seamless.

As you suggested, @bluefirestorm, there are workarounds to this(Unfortunately, Docker for Windows still relies on Hyper-V/WSL2 backend). I don't want to go there yet because WSL2 is a great tool that I still like to use. Are there still actions that we can take here? (I will be trying a a fresh Windows 10 VM in the meantime)

I am wondering also if this is a bug in VMWare's side?

Sidenote:

As a workaround, I migrated this VM to hyper-v-based virtualization and everything works just fine. I am not happy with hyper-V though. VMWare's buttery smooth 60fps 4k resolution when using the VM is a treat.

[martopad]

Followup:

I tried your suggestion, @bluefirestorm, and everything works... **bleep**...

Tested on:

1) windows 11 evaluation for VMWare.

2) Windows 10 pro 21H2.

NOK case:

1) Windows pro 1909 build.

looks like there is an issue with windows 1909....

Hmmmm the Guest OS build does matter....

[martopad · Accepted solution]

Hello. The problem was the 1909 guest image I was running-- It was too old. So I got a 20H2 Windows 10 Enterprise image, and this problem is gone!

Thanks, @bluefirestorm, and @continuum, for the help!