Windows 10 VMs black screen with Ryzen 5800x host

[Barreleye]

I'm using Player version 16.1.2 on my new Ryzen 5800x build, and not only do my existing VMs I created on an Intel i5-4670 no longer work properly, I can't create new ones, because they lock up during Windows setup, some time after I've entered the product key. So it just doesn't work at all with Windows 10 Pro x64 host under Ryzen 5800x. Windows Sandbox and WSL2 do work fine, and they worked fine on my old Intel system, for which VMware Player worked fine with several Windows 10 Pro 32 and 64 VMs. On the new system, my existing Kali and FreeDOS VMs do work fine.

For my existing Windows 10 VMs, I have no password, and they normally automatically log on. For the Ryzen build, this happens, and I see my desktop, but then the VM window goes black, and it shrinks to a smaller size. Then it goes through a cycle where it appears to be logging me in again, complete with lock screen and spinning circle, to my desktop, to a black screen, and then a black screen surrounded on the bottom and sides with a thick grey border, and then it repeats ad infinitum. When the desktop is visible, I can sometimes interact with it before the screen goes black, but it doesn't work right. For example, right-clicking to bring up a context menu might show just the first column of letters in the menu, and nothing works.

I've read about downgrading the VMware SVGA adapter to the Microsoft VGA one, but that requires booting into Safe Mode, which I have been singularly unable to do. I can't get F8 to register during reboot.

Any ideas?

[Barreleye]

My VMware VMs do work fine as long as I disable all the Windows 10 Hypervisor features, but VMware Player was supposed to have been made compatible with them somewhere in the v15 series, and indeed it was on my i5-4670. I'm using 16.1.2 with my 5800x, and it's back to being incompatible with Microsoft's hypervisor. Any ideas?

[Barreleye]

The latest version I just downloaded and installed, 16.2.0, sadly does not fix this problem. Even more sadly, it removes dark mode from player settings.

[wila]

Hi,

Try disabling VMware Tools auto upgrade mode, it might be that that got stuck in a loop somehow.

re. lost Dark Mode support.
Yep, known bug in 16.2

--
Wil

[Barreleye]

It's set to "Use application default (currently update manually)."

Do you think Asus AI Suite III could be the problem? If I ever get a spare afternoon I feel like messing with this, I might try uninstalling it, but with the full expectation I'll be restoring from an image backup.

[wila]

Hi,

In the past there has been issues with some of ASUS specific tools in combination with VMware Workstation.

Had to search for a bit, this thread: https://communities.vmware.com/t5/VMware-Workstation-Pro/Workstation-Pro-vthread-Crash/m-p/1399650

Probably not related to your issue though.

--
Wil

[Barreleye]

This sounds like my problem, but no solution unfortunately.

https://www.reddit.com/r/vmware/comments/kl9xah/vmware_workstation_16_ryzen_5000_windows_10_guest/

[wila]

Hi,

The Hyper-V host support is a bit troublesome to put it mildly.
It is a feature that seems to leapfrog all the time.
When Microsoft releases a new update for their OS, it breaks again and VMware has to spent time on fixing it. That must be super frustrating for VMware engineering.

It certainly is frustrating for VMware's customers who depend on it. It starts be speculation worthy, as it is not as if Microsoft delivers a competing product. (sigh)

Even getting the Hyper-V features disabled at the host is troublesome.
Look at all the steps required at this KB article: https://kb.vmware.com/s/article/2146361

Yes all those steps, not just unticking the checkbox.

--
Wil

[Barreleye]

At this point, I'd just be interested to hear whether or not other Ryzen 5800x and similar users have the problem I described with the Hyper-V features enabled. I found the one on Reddit I linked to that does, but it would be interesting to determine if it's a widespread problem or there's something unique (and unknown) about our systems.