VMware 16.2.1: Display 3D acceleration not working, Windows 10 Guest runs choppy

[UlrichMeier]

VMware Workstation unrecoverable error: (mks)
ISBRendererComm: Lost connection to mksSandbox (2878)
A log file is available in "/data/srv/vm/Win10_pro_L3/vmware.log".
You can request support.

To collect data to submit to VMware support, choose "Collect Support Data" from the Help menu.
You can also run the "vm-support" script in the Workstation folder directly.
We will respond on the basis of your support entitlement.

on Ubuntu 21.10, Lenovo X1 Yoga 6th Gen, 32 GB RAM, 2 TB HD

If 3d accel is disabled, it works.

However - the Windows 10 guest is choppy.

[ivanbondar]

Same issue here. Running Arch Linux kernel "5.14.16-arch1-1", video is "VGA compatible controller: Intel Corporation UHD Graphics 620 (rev 07)", Vmware Workstation ver. 16.2.1 build-18811642. The issue started after update fo 16.2.1

There are two workarounds I've found so far:

1. Disable 3D acceleration in VM
2. Use Xorg instead of Wayland. In Xorg everything works fine even with 3D acceleration enabled.

[banackm]

You could try setting these config options:

mks.enableX11Presentation=TRUE
mks.enableVulkanPresentation=FALSE

That will force us to draw the window using X11 instead of Vulkan, which is normally less performant, but might work better on some setups?

If someone can post a vmware.log and mksSandbox.log where this is hitting our graphics team can take a look.

[UlrichMeier]

some sandbox logs - hope it helps

I use Wayland and I need to use Wayland...

[ivanbondar]

Here are my logs as well. I hope they will help to find out the root cause.

[banackm]

Yeah, you're hitting a crash in the Vulkan window code.  I would definitely try those config options I posted.  They should work even on Wayland hosts, it'll just use Wayland's X11 emulation layer.

[ivanbondar]

@banackm Thank you very much for your fast reaction and issue analysis!

The only problem in my  case is that the VM is encrypted (I have to use virtual TPM there) and adding options to the vmx file is not so easy 😞

Anyway, I hope this issue will be fixed in the future release. At this time maybe a downgrade to 16.1 will be a better option.

[banackm]

You can also put the options in ~/.vmware/config instead, and it'll load them from there for all your VMs.

[ivanbondar]

I confirm both options saved in ~/.vmware/config did the magic and now my VM boots under Wayland with 3D acceleration enabled.

Thanks a lot, @banackm !

Last question from my side - "That will force us to draw the window using X11 instead of Vulkan", does this config use OpenGL or it is a software rendering? My impression is that the 3D rendering almost as slow as without 3D acceleration, but maybe it's just a perception...

[banackm]

That configuration will change how we draw the final contents into the window, but not how the actual graphics rendering is done.

So on your card, the rendering is still being done on your GPU with Vulkan, and then when we have to draw into the window we will use X11 for the last step (usually called "Presentation", hence the config option name).

The cost is proportional to the display size and the frame rate of the workload, so how much of a performance hit that is will depend on your system and workload.  Applications that are running very graphics heavy workloads and already getting low frame-rates probably won't notice at all, whereas as the graphics workload gets lighter and the resolution/FPS increases, there will start being more of a difference between the two modes.

On my system, I can't tell the difference between the two paths up until my graphics workload gets too high and my host GPU gets saturated, and then my performance falls off a cliff.  So if your application has a frame-rate limiting option, you could try lowering the FPS and ironically your performance might improve.  But the difference between the two paths might be greater on lower-end GPUs?

[ivanbondar]

Thank you for the comprehensive explanation, @banackm ! 

[UlrichMeier]

did that and it works so far.

Thx!!

[camtt]

Thanks for this. Those parameters got 3D OpenGL support working again for me on Gentoo with a pre Turing Nvidia GTX1060.

I was back on Workstation 16.1 until I saw this fix.

[Dani_LT]

Sorry for my bad English, but I have the same problem with VM Workstation 16.2.0 with a Win10 host and Win7 guest. When I start a 3d game in the Win7 guest, it crashes with the same error message. I went back to version 16.1.2. Is this possibly a general problem with 3d acceleration in version 16.2.x?

[Parazythum]

Hi, thanks for the solution. I'm using Ubuntu 18.04 with a NVidia GTX 1050 on my laptop.

I added the 2 lines and my VMs started again.

BUT there is a big performance drop with 3D. To test 3D/OpenGL, I'm using under my Windows 7 guest the "GPU CAPS Viewer" utility, and where some tests showed hundreds of FPS before with v16.1.2, now it goes from 30 to 80 FPS only 😞   --- I'm using a VM just for Photoshop, and some functions are now a bit choppy sometimes, so 3D/OpenGL support is essential to me. Or Vulkan if it could work !

Speaking of it : I DO have the vulkan functions enabled (and tested), but setting mks.enableVulkanPresentation=TRUE just results in a crash.

[CollegiateComp]

@banackmIs this issue caused by something that needs to be addressed on the VMware side or something that we should be looking to resolve on the host OS side? I'm experiencing the same issue with a NVIDIA GeForce RTX 2080 Super (non-free drivers) and Intel(R) Core(TM) i9-10885H implemented with Nvidia Prime, but from what I read from the release notes for 16.2.1 I should be able to use the Vulkan renderer support given my GPU's turing architecture (unless I'm misunderstanding). While a work around is definitely great to avoid having to roll back to 16.1, I'm just curious if this is something Linux users should be looking to resolve ourselves if the issue is not with Workstation 16.2.

[Parazythum]

OK I found the solution ! The problem came from the host, in my case :

    Ubuntu 18.04 + Integrated Intel Graphics i915 + NVidia GTX 1050 (it's a laptop)

Vulkan happens to run by default on the 1st GPU it lists. For me, it was the Intel graphics integrated GPU, which of course has no Vulkan support...

You can test Vulkan support on your system by using the executable "vkcube", installed with a package named "vulkan-tools". I used a well-known PPA to install the v460 nvidia proprietary drivers, from here : https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa . This utility simply displays a cube. If your system is not configured properly, the little window will be all black, with no cube.

To force Vulkan to run on the desired GPU, I used this command line for my NVidia card :

        export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"

You can list the .json files in /usr/share/vulkan/icd.d/ to find the best one for your system (Intel, AMD...)

After this command was launched : oh joy, the cube was finally displayed with "vkcube".

And using it before launching VMWare Workstation is good ! The 3D/OpenGL performance is total.

To permanently set this export, you can add the command line in your .profile. Log out, log in, launch your VM with these options in your .vmx file :

        mks.enableX11Presentation=FALSE
        mks.enableVulkanPresentation=TRUE

... et voilà !

[CollegiateComp]

Parazythum's solution resolved the issue on my machine as well. One potentially useful discovery I made while testing this is that Vulkan and Workstation's 3D acceleration work properly on my Optimus laptop when Nvidia Prime is set to "on-demand" and Intel integrated graphics is driving the display. I only encounter this issue when Nvidia Prime is set to "performance mode" and the discrete Nvidia GPU is driving the display, in which case adding export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json" to ~/.profile resolves the issue.

[UlrichMeier]

in Lenovo X1 6th gen yoga with Intel GPU and Ubuntu 21.10 using Wayland this does not work. (vkcude showing the cube rotating)

The other workaround works.

[banackm]

This issue would be specific to Linux, but there may be other issues that hit on Windows.  You may want to follow the thread here for a similar sounding issue that hits on Windows hosts: https://communities.vmware.com/t5/VMware-Workstation-Pro/VM-Crash-16-2-1-build-18811642/td-p/2877469