VMWare Workstation 16: sluggish UI at 4K (Windows 10 host and guest)

[metroidsnes]

Hi, I get sluggish UI (slow alt-tab, slow scrolling in some programs) and screen tearing when playing Youtube videos on a 4K display (Windows 10 host and guest). My machine: AMD Ryzen 9 3900X 12-core, 64 GB ram, NVMe drive, GeForce RTX 3070Ti. VM has assigned 12 cores (out of 24), 16 GB ram and 8 GB video memory. What’s the bottleneck and is there anything that can be done to improve responsiveness; I mean changes to hardware and/or the software.

Asking differently; is there a hardware I could buy to run at 4K smoothly?

[wila]

Hi,

Please open the vmware.log in your VM's folder and search for "Monitor Mode:"

What does it say after those 2 words?

--
Wil

[metroidsnes]

Monitor Mode: CPL0

[wila]

Hi,

Ok, so it is not because you are running your VM under Hyper-V because then it would have said ULM.
Hopefully someone else has an idea about what to try.

--
Wil

[bluefirestorm]

Since you mentioned that YouTube video tearing on both host and guest, it is likely a hardware issue and/or video/colour setting issue. And unlikely to be related to VMware Workstation. For hardware, unlikely to be the GPU as it is an Ampere card (most likely it has DP1.4, HDMI 2.0 or even HDMI 2.1), so the hardware in question here are the adapter and the monitor itself.

You could try to go to some other forum such as /r/monitor on reddit but most likely you will face these questions about your setup.

The questions:
What is the connection from GPU to the monitor? HDMI or DP? Is there an adapter involved such as USB-C to HDMI or DP to HDMI? Some of these adapters can only handle 30Hz refresh rate at 2160p resolution.
What is the spec of the HDMI (1.4 or 2.0) or DP (1.2 or 1.4) at the monitor side?

For software settings, the colour settings can also affect the maximum refresh rate. Higher colour settings mean more data, more data means more bandwidth required; which would then affect what connection type is required for at least 60Hz refresh rate.

For 60Hz 2160p, it is best to have HDMI2.0 or DP1.2 or newer at the monitor side. For G-Sync compatible (assuming the monitor has AMD FreeSync), DP 1.2 (or newer) connection is required.

[metroidsnes]

Oh, it's not what I meant. The screen tearing only happens on the guest. On the host everything is butter-smooth. Both the guest and host are running Windows 10. Sorry for the confusion, I should have written that differently.

[bluefirestorm]

You could try reducing the amount of VM display memory from 8GB to 2GB. It is counterintuitive. That memory won't be using the GPU memory; it will be taken out of CPU RAM. I don't think the nvidia-smi tool on Windows shows the GPU memory usage per process (it shows N/A). Having too big RAM allocated for a process might cause some pages to be swapped out by the OS (to slower SSD/HDD paging file) and cause it to be swapped back in when it is needed and thus slow things down.

I use Workstation Pro on an Ubuntu host. I run VMs in a window with a 16:10 aspect ratio at a 3072x1920 custom resolution on a 27" 2160p 60Hz display. With this setup I find I don't have to use scaling on both host and guest VMs (be it Windows or Ubuntu). The YouTube videos that I watch are usually 1080p and I would stream it 21.4on a second monitor on the host.

On a 27" 2160p display on Windows 10 host it is bit too tiny for my eyes which would then require scaling. Scaling (on host and/or guest) might also be a factor for sluggishness. At least on Ubuntu Gnome, it warns against "fractional scaling" that it can "lower speed and reduce sharpness". Fractional scaling would be 125%, 150% instead of whole number such as 200% or 300%.

Going back to the nvidia-smi, the mksSandbox process for the VMs rarely show GPU memory usage above 400MB even I though I specify 1GB on the VM display settings. I just did a playback of a 4K movie trailer on YouTube with both VM and browser on full screen 2160p and there are no tearing and the GPU memory usage of mksSandbox processs hovers between 500-600MB. Without naming the movie, it has high speed action sequences, so if there was any tearing it would have been obvious on the action sequence as oppose to watching a 4K video of a giant tortoise in the Galapagos.

So apart from the VM display memory setting and avoiding host/guest fractional scaling, I don't have any other ideas as your setup is different even though I do use a 4K display.

[metroidsnes]

Thanks for all this info. I tested the scaling and video memory settings but with no luck; the guest is as much sluggish as it was before, screen tearing included.

How I tested (27" 4K monitor):
1. Run VM. Change scaling from 175% to 100%. Check system responsiveness. Then change scaling also on the host from 175% to 100%. Result: no improvement in responsiveness on the guest (also when using exclusive mode).
2. Change scaling back to 175% on the guest and the host. Shutdown guest. Change video memory from 8GB to 2GB. Start the guest and test. Result: no improvement.

[wila]

One more question, as you're on a Windows host & running a Windows guest.. there's still a chance that you are using Unity. Unity is known to be slow.
The best GPU performance tends to be when the guest is full screen.

--
Wil

[metroidsnes]

Whether I run the VM fullscreen or not doesn't seem to make any difference. When in fullscreen, Unity view is grayed-out. Another thing that I noticed is that when I run a game inside a VM (not very demanding one) it's seems to be capped to 30 FPS.

[declif1]

Please check which graphics card is used. Even if only one graphics card is installed, microsoft.software.renderer may be used. As described in the following article, this can be changed as follows:

https://communities.vmware.com/t5/VMware-Workstation-Pro/Make-VMWare-Workstation-use-my-Nvidia-card/...

Add the following lines to the vmx file (vendorID=nvidia!):

mks.dx11.driverType = "unknown"
mks.dx11.vendorID = 0x10de