Enabling 3D Acceleration decreases performance on Linux guest

[reijin]

Hi!

I use a Windows 10 x64 host with an Intel HD 620 Chip. Performance within my Windows guests is fine, but with my Linux guest (Xubuntu 17) I can only watch 720p Youtube videos.

To see how I can improve this I ran some tests: Enabling/disabling Vsync; allocating more VideoRAM; En/Disable 3D acceleration.

With disabled 3D Acceleration I get the best performance in terms of video and glxgears (around 1000FPS). With enabled 3D acceleration, Chrome enables several hardware rendering features, but performance is still worse. In glxgears, the best results with enabled 3D acceleration is around 400FPS.

In all cases, direct rendering seems to be enabled.

➜  ~ glxinfo | grep direct

direct rendering: Yes

    GL_ARB_direct_state_access, GL_ARB_draw_buffers,

    GL_ARB_draw_indirect, GL_ARB_draw_instanced, GL_ARB_enhanced_layouts,

    GL_ARB_map_buffer_range, GL_ARB_multi_bind, GL_ARB_multi_draw_indirect,

➜  ~ glxinfo | grep OpenGL

#OpenGL vendor string: VMware, Inc.

OpenGL renderer string: llvmpipe (LLVM 5.0, 256 bits)

OpenGL core profile version string: 3.3 (Core Profile) Mesa 17.2.8

OpenGL core profile shading language version string: 3.30

OpenGL core profile context flags: (none)

OpenGL core profile profile mask: core profile

OpenGL core profile extensions:

OpenGL version string: 3.0 Mesa 17.2.8

OpenGL shading language version string: 1.30

OpenGL context flags: (none)

OpenGL extensions:

OpenGL ES profile version string: OpenGL ES 3.0 Mesa 17.2.8

OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00

OpenGL ES profile extensions:

Open-vm-tools are installed:

➜  ~ dpkg -l | grep open-vm-tools

ii  open-vm-tools                         2:10.1.10-3ubuntu0.1                     amd64        Open VMware Tools for virtual machines hosted on VMware (CLI)

ii  open-vm-tools-desktop                 2:10.1.10-3ubuntu0.1                     amd64        Open VMware Tools for virtual machines hosted on VMware (GUI)

ii  open-vm-tools-dkms                    2:10.1.10-3ubuntu0.1                     all          Open VMware Tools vmxnet kernel module (deprecated)

I use this VM as my daily work machine as part of a security concept, where I only use Linux for day to day usage. But watching videos is basically impossible with this. Does anyone of you have a hint how I can improve the performance of the Linux guest? Could there be a problem with Open-vm-tools?

The Windows guests run pretty smooth, so I wonder what the problem is....

[bluefirestorm]

There are only 6 Kaby Lake CPUs that has Intel HD 620 graphics chip. They are all mobile U series.

https://ark.intel.com/compare/95451,97466,95443,97472,95442,124977

So you have either a laptop or a mini-PC which makes getting a discrete graphics card impossible or ridiculously expensive (assuming the system has a Thunderbolt 3 port, a Thunderbolt 3 enclosure without a GPU probably costs as much or even more than the PC itself).

The U series of CPUs can throttle down significantly when battery is being used (in the case of a laptop) or when the system generates too much heat. How aggressive the throttling down depends on the system manufacturer.

Since these CPUs are all 2 core/4 thread, suggest that you keep the VM configuration to only 1 processor and 2 cores per processor (i.e. max 2 vCPU).

Assuming that you are using version 12.5.x or 14.x, you could try adding the following lines to the vmx configuration file of the Xubuntu VM

mks.enableDX11Renderer = "FALSE"

mks.enableGLRenderer = "TRUE"

This will switch Workstation to use OpenGL instead of using the default DX11 for Windows host when 3D acceleration is enabled for a VM.

[reijin]

Ok! First of all thanks a lot for the quick reply!

Here are some comments:

So you have either a laptop or a mini-PC which makes getting a discrete graphics card impossible or ridiculously expensive (assuming the system has a Thunderbolt 3 port, a Thunderbolt 3 enclosure without a GPU probably costs as much or even more than the PC itself).

The U series of CPUs can throttle down significantly when battery is being used (in the case of a laptop) or when the system generates too much heat. How aggressive the throttling down depends on the system manufacturer.

Yes and yes. I'm fully aware of this and you are correct.

Since these CPUs are all 2 core/4 thread, suggest that you keep the VM configuration to only 1 processor and 2 cores per processor (i.e. max 2 vCPU).

I read about this in an other thread when I set up a few weeks ago. So this is already implemented. Thanks for noting it!

Assuming that you are using version 12.5.x or 14.x, you could try adding the following lines to the vmx configuration file of the Xubuntu VM

mks.enableDX11Renderer = "FALSE"

mks.enableGLRenderer = "TRUE"

This will switch Workstation to use OpenGL instead of using the default DX11 for Windows host when 3D acceleration is enabled for a VM.

I'm running 14.x actually, but the VM is of ESXi 6.5 compatibility. I will try the suggested settings. I wonder what the rationale is though, as my host is a Windows machine and should suppport DX11 better, or what exactly is this setting supposed to achieve?

Edit:

The results are in! I enabled and disabled the following flags respectively:

mks.enableGLBasicRenderer = "X"

mks.enableGLRenderer = "X"

mks.enableDX11Renderer = "X"

mks.enableD3DRenderer = "X"

I also set the vram to:

svga.vramSize = "134217728" --> 128MB

D3D only:

4645 frames in 5.0 seconds = 928.976 FPS

5063 frames in 5.0 seconds = 1012.496 FPS

4804 frames in 5.0 seconds = 960.584 FPS

5167 frames in 5.0 seconds = 1033.347 FPS

DX11 only:

2374 frames in 5.0 seconds = 473.952 FPS

2601 frames in 5.0 seconds = 519.829 FPS

2495 frames in 5.0 seconds = 498.874 FPS

2348 frames in 5.0 seconds = 469.470 FPS

1620 frames in 5.0 seconds = 323.355 FPS

OpenGL only:

2825 frames in 5.0 seconds = 563.627 FPS

3448 frames in 5.0 seconds = 689.532 FPS

3237 frames in 5.0 seconds = 647.236 FPS

3417 frames in 5.0 seconds = 681.149 FPS

OpenGL (no base):

2660 frames in 5.0 seconds = 531.744 FPS

3420 frames in 5.0 seconds = 683.936 FPS

3090 frames in 5.0 seconds = 617.643 FPS

3008 frames in 5.0 seconds = 600.385 FPS

3127 frames in 5.0 seconds = 625.276 FPS

In all cases Chrome also didnt recognize any hardware capabilities and switched to software rendering. Enforcing hardware acceleration in Chrome always lead to worse performance. Test video was a FullHD 60fps Youtube video.

While D3D seems to yield the best results, it is still very similar to disabled 3D rendering. This is really odd..

Cheers!

[bluefirestorm]

but the VM is of ESXi 6.5 compatibility

I take it you mean the hardware compatibility is version 13. The minimum to support OpenGL 3.3 core profile in a Linux VM and OpenGL 3.3 core profile/DX10 in a Windows VM is version 12. Try setting it to version 12 or 14.

I wonder what the rationale is though, as my host is a Windows machine and should suppport DX11 better, or what exactly is this setting supposed to achieve?

I found that switching to OpenGL in Workstation 12.5.x with an Nvidia GPU on a Windows 10 host gave prepare graphics capability to the guest. One notable difference is the support of supersampling when OpenGL is used. It also fixed an odd bug where the Chrome tabs were not rendered properly inside the VM when Chrome accelerated graphics is enabled (which is by default). It was this Chrome bug that made me switch. But it might be different for an Intel integrated graphics.

You can try to compare the vmware.log there are 243 properties that look like this

vmx| I125:   cap[ 73]: 0x00000001 (SUPERSAMPLE)

An alternative is to look at the chrome://gpu properties inside the VM.

The results are a bit odd. What is the version of vmwgfx driver that you have inside the Xubuntu VM? dmesg | grep vmwgfx

[reijin]

I take it you mean the hardware compatibility is version 13. The minimum to support OpenGL 3.3 core profile in a Linux VM and OpenGL 3.3 core profile/DX10 in a Windows VM is version 12. Try setting it to version 12 or 14.

Correct. Last time I tried to upgrade my VM lead to it not booting anymore. I will try setting level 14 in the vmx and see what happens. Looking at the output of glxinfo | grep OpenGL tells me that OpenGL 3.3 seems to be active (see above), or am I missing something?

You can try to compare the vmware.log there are 243 properties that look like this

vmx| I125:   cap[ 73]: 0x00000001 (SUPERSAMPLE)

An alternative is to look at the chrome://gpu properties inside the VM.

the Chrome properties are the place where I checked to see if acceleration was enabled or not. It was disabled in most of the cases and even if it was enabled, the performance was still lower than software rendering. In all cases I tried to override the settings by disabling the gpu blacklist (in the chrome://flags section). Is there any more info that could be of use from the log?

The results are a bit odd. What is the version of vmwgfx driver that you have inside the Xubuntu VM? dmesg | grep vmwgfx

➜  ~ dmesg | grep vmwgfx

[    2.915922] [drm] Initialized vmwgfx 2.13.0 20170607 for 0000:00:0f.0 on minor 0

The date there seems to indicate a recent version.

edit:

I re-ran the tests with a fresh installed Xubuntu VM with installed open-vm-tools and this time compatibility version 14. Again, with enabled 3D acceleration the framerate for gfxgears drops significantly (~300FPS compared to ~1000FPS). This leads me to the conclusion that the compatibility level is not the issue here.

edit2:

More news. Apparently D3D is the fastest as it leads to software rendering only. This becomes clear when looking at the vmware.log. The shortened logs with different settings are here:

vmwarelog - Pastebin.com

tl;dr: 3D enabled will lead to capabilities being enabled, but low performance (300FPS). D3D will lead to all caps being disabled (highest performance with 3D enabled) and 3D disabled disables all caps. So it seems the capabilities somehow can't be used to their full potential....

edit3:

In my desperation I tried to update the Intel Drivers, but as it turns out the official drivers are incompatible with my Lenovo laptop and require an installer from the vendor. So I decided to enable the mks.gl.allowBlacklistedDrivers = "TRUE" setting, but it doesn't seem to have an effect.

I'm running on FullHD resolution on a UHD screen - so it is being upscaled. My Windows VMs just seem to have better performance, but in reality when watching a FullHD 60FPS video they struggle as well. Overall scrolling and mouse movement is much smoother when using Windows though. The main OS (Win10x64) is fully capable of showing even 4K video at 60FPS.

This is just frustrating, as I want to use the Linux VM as my daily machine, but the sluggish video performance breaks the immersion

I left Virtualbox because of all those nuisances and now I'm using a professional product, which seems to struggle with the same stuff. It's quite surprising.

[bluefirestorm]

I did some glxgears run on an Ubuntu 16.04 VM (with virtual firmware EFI, 1440x900 resolution, windowed mode) on Workstation 12.5.9. The headscratcher is the vmwgfx seems to be stuck at 2.9.0 20150810 no matter I use open-vm-tools or manually install VMware Tools 10.2.0. I made the configuration to use the HD530 on the i7-6700HQ instead of the Nvidia. Full HD (1920x1080) is not in the Ubuntu VM resolution selection. I know I can create 1920x1080 but I am just too lazy at the moment :smileylaugh: My laptop screen can only handle 1920x1080 and I don't have scaling turned on the Windows 10 1709 host.

without 3D acceleration: around 7xx fps

3D acceleration, DX11: 8xx-9xx fps

3D acceleration, OpenGL; 13xx-14xx fps

So perhaps the problem lies in the screen resolution/scaling of the guest VM or host or both. Is scaling turned on at the Windows 10 host? How about scaling at the xubuntu guest VM? I do not know if version 14 has "Display Scaling" checkbox in the VM for non-Windows guests. For version 12.5.x it seems to be available only for Windows guests but the vmx entry would be

gui.applyHostDisplayScalingToGuest = "TRUE"

if it is checked.

[reijin]

Thanks for sticking with me!

I ran some tests, too and upgraded my mesa driver to 18.0.0.rc4. OpenGL renderer performance is much better now (around 800FPS), but still below the non-3d mode. Also I noticed the following lines in my vmware.log:

2018-02-25T16:15:37.976+01:00| mks| W115: DX11Renderer: SDK=7, driver=unknown, feature level=11.0

(DX11 renderer)

2018-02-25T15:54:58.805+01:00| svga| I125: GLManagerCheckCapabilities: Detected buggy Intel GPU driver on Windows.

(OpenGL renderer)

Also I noticed, that Vmware seems to try to find my settings from the wrong location (tries in AppData/Roaming, but there is not config.ini). So maybe my setting of mks.gl.allowBlacklistedDrivers = “TRUE” didn't really work.

So perhaps the problem lies in the screen resolution/scaling of the guest VM or host or both. Is scaling turned on at the Windows 10 host? How about scaling at the xubuntu guest VM? I do not know if version 14 has "Display Scaling" checkbox in the VM for non-Windows guests. For version 12.5.x it seems to be available only for Windows guests but the vmx entry would be

gui.applyHostDisplayScalingToGuest = "TRUE"

Indeed I remember I did some changes to this, as I wanted my machine to stay FullHD on my UHD display. I have DPI scaling enabled on my host, as the GUI of some applications would be unreadable otherwise. In WS 14 there is a setting to set a fixed screen resolution for a VM in the GUI, but since I suspected it could be one reason for my issues, I removed it. I disabled scaling in the guest VM. I'm not sure if setting applyHostDisplayScalingToGuest is what I want, since I DON'T want my VM to scale to UHD as I don't need it and High DPI displays are poorly supported on Linux. What do you suggest I should do?

edit:

Ok, maybe we are on to something. I disabled DPI scaling for VMWare WS (compatibility tab and switched to override... System). Now the GUI is unscaled. Interestingly this seems to have impacted performance for the DX11 renderer, as now it is at around 700-900 FPS. Buuuuut, this had negative effects on my mouse detection, as it is all over the place for split seconds. Single clicks are ok, but selecting text leads to wrong selections, as the mouse coordinates seem to be detected completely elsewhere. I set my Input settings from Automatic to Alyways and Never, but this didnt really change much. The performance of the OpenGL renderer seems to have suffered from this setting, as the FPS dropped to around 700 now. I feel like we could be on to something, but I'm not sure what I could try now....

[bluefirestorm]

I have pasted below the GLManager entries in the vmware.log with the HD530. It also has the buggy Intel GPU driver line. This will be a slightly more friendly albeit truncated version of the SVGACaps 0 - 243 array of values.

I think the display scaling is for VMware Workstation to automatically adjust the scaling of the guest if it detects a guest with high DPI display (which can be annoying, aside from the fact not all applications scale well or only have partial scaling). The GUI check box is available for a Windows 10 VM in the Display setting but not available for Linux guests so it might not work automatically either for a Linux guest.

If I am not mistaken, the allowBlacklistedDrivers is for Workstation on Linux hosts. Workstation on Linux hosts also use OpenGL as the render engine when 3D acceleration is enabled for the VM.

As for the mouse behaving erratically, you could try setting the Preferences in Input. There should be a setting called "Optimize for Games" and set it to "Never".

| mks| I125: GLManager: Shader Version 4.00 supported

| mks| I125: GLManager: Host supports 1 dual-source blend draw buffers.

| mks| I125: GLManager: Host supports 16 viewports.

| mks| I125: GLManager: Using sampler objects.

| mks| I125: GLManager: Using texture views.

| mks| I125: GLManagerCheckCapabilities: Detected buggy Intel GPU driver on Windows.

| mks| I125: GLManager: Host supports 84 uniform buffers 14 vertex blocks 14 fragment blocks and 14 geometry blocks

| mks| I125: GLManager: Host supports 65536 byte uniform blocks.

| mks| I125: GLManager: Uniform buffer offset alignment is 16 bytes.

| mks| I125: GLManager: Host supports 1024 total geometry output components.

| mks| I125: GLManager: Host supports 256 geometry output vertices.

| mks| I125: GLManager: Host supports 4096 geometry uniform components.

| mks| I125: GLManager: Host is capable of HQ line stipple emulation with 204 vertices per draw.

| mks| I125: GLManager: Emulating point sprites with quads.

| mks| I125: GLManager: using compatibility profile

| mks| I125: GLManager: maxVertAttrStride = 2048

| mks| I125: GLManager: Host supports 32 vertex texture units, 32 fragment texture units, 32 geometry texture units and 192 combined texture units.

[reijin]

I have pasted below the GLManager entries in the vmware.log with the HD530. It also has the buggy Intel GPU driver line. This will be a slightly more friendly albeit truncated version of the SVGACaps 0 - 243 array of values.

I think the display scaling is for VMware Workstation to automatically adjust the scaling of the guest if it detects a guest with high DPI display (which can be annoying, aside from the fact not all applications scale well or only have partial scaling). The GUI check box is available for a Windows 10 VM in the Display setting but not available for Linux guests so it might not work automatically either for a Linux guest.

If I am not mistaken, the allowBlacklistedDrivers is for Workstation on Linux hosts. Workstation on Linux hosts also use OpenGL as the render engine when 3D acceleration is enabled for the VM.

As for the mouse behaving erratically, you could try setting the Preferences in Input. There should be a setting called "Optimize for Games" and set it to "Never".

My vmware.log seems to contain the same information like yours. I removed the allowBlacklistedDrivers option, as it didn't really have any effect. Regarding the input: With the disabled DPI scaling setting it to never didn't yield any improvement. Overall it doesn't really matter though, as it seems none of the settings seem to have a noticable (positive) effect.

Your previous mention of running in windowed mode made me try to compare Fullscreen with Windowed.

I setup a new Xubuntu VM with 16.04.3 mesa 17.2.8 and vmwgfx 2.13.0 (3D acceleration enabled - default renderer) and:

ii  open-vm-tools                          2:10.0.7-3227872-5ubuntu1~16.04.2          amd64        Open VMware Tools for virtual machines hosted on VMware (CLI)

ii  open-vm-tools-desktop                  2:10.0.7-3227872-5ubuntu1~16.04.2          amd64        Open VMware Tools for virtual machines hosted on VMware (GUI)

ii  open-vm-tools-dkms                     2:10.0.7-3227872-5ubuntu1~16.04.2          all          Open VMware Tools vmxnet kernel module (deprecated)

Window scaling in Windows is enabled again, so the UI is being upscaled for the UHD display. Results:

Renderer/HighDPIScaling (Host OS Windows 10)	Yes	No
Software (Fullscreen)	1408 frames in 5.0 seconds = 281.498 FPS 1371 frames in 5.0 seconds = 274.190 FPS 1557 frames in 5.0 seconds = 310.712 FPS	5842 frames in 5.0 seconds = 1168.134 FPS 4898 frames in 5.0 seconds = 979.350 FPS 5064 frames in 5.0 seconds = 1012.744 FPS 5130 frames in 5.0 seconds = 1025.875 FPS
Software (Window)	6305 frames in 5.0 seconds = 1260.871 FPS 5528 frames in 5.0 seconds = 1105.467 FPS 5476 frames in 5.0 seconds = 1095.108 FPS -->Significant drop in Fullscreen	5206 frames in 5.0 seconds = 1041.125 FPS 5094 frames in 5.0 seconds = 1018.746 FPS 5276 frames in 5.0 seconds = 1055.030 FPS 5387 frames in 5.0 seconds = 1077.386 FPS
OpenGL (Fullscreen)	5208 frames in 5.0 seconds = 1041.516 FPS 5402 frames in 5.0 seconds = 1080.360 FPS 4815 frames in 5.0 seconds = 962.930 FPS 4622 frames in 5.0 seconds = 924.371 FPS	3761 frames in 5.0 seconds = 752.011 FPS 3964 frames in 5.0 seconds = 792.780 FPS 3562 frames in 5.0 seconds = 711.826 FPS 3646 frames in 5.0 seconds = 729.182 FPS
OpenGL (Window)	3728 frames in 5.0 seconds = 745.588 FPS 3742 frames in 5.0 seconds = 744.684 FPS 3722 frames in 5.0 seconds = 744.363 FPS 3739 frames in 5.0 seconds = 747.755 FPS -->It is reversed here...?!	3677 frames in 5.0 seconds = 735.377 FPS 3681 frames in 5.0 seconds = 735.987 FPS 3712 frames in 5.0 seconds = 742.333 FPS 3712 frames in 5.0 seconds = 742.337 FPS
Default (3D enabled; Fullscreen)	4640 frames in 5.0 seconds = 927.890 FPS 4862 frames in 5.0 seconds = 972.278 FPS 4683 frames in 5.0 seconds = 936.577 FPS 4915 frames in 5.0 seconds = 982.908 FPS	3883 frames in 5.0 seconds = 775.419 FPS 4066 frames in 5.0 seconds = 811.471 FPS 3931 frames in 5.0 seconds = 785.194 FPS 3988 frames in 5.0 seconds = 797.060 FPS
Default (3D enabled; Window)	5110 frames in 5.0 seconds = 1021.874 FPS 5122 frames in 5.0 seconds = 1024.300 FPS 5714 frames in 5.0 seconds = 1142.725 FPS 5244 frames in 5.0 seconds = 1048.649 FPS -->Slightly better in Windowed	5126 frames in 5.0 seconds = 1025.110 FPS 5207 frames in 5.0 seconds = 1041.315 FPS 5343 frames in 5.0 seconds = 1068.329 FPS 5110 frames in 5.0 seconds = 1021.741 FPS

For me there seems to always be a ceiling of around ~1200FPS....

Another thing I've noticed: When 3D is enabled, the GPU usage actually goes up, but CPU and GPU usage reach the same level (around 60% during benchmark). When Software rendering is enabled, the GPU is not stressed and has around 3% utilization in the Windows Taskmanager. How does this behave for you?

[bluefirestorm]

I installed Workstation Player 14.1.1 on a Surface Pro 3 with i5-4300U, HD 4400 (200MHz base graphic clock vs the 300MHz for the HD 620). The SP3 was plugged in, on a Surface Pro dock, and the Power Options was set to "High Performance". I created a Xubuntu 17.10.1 VM with virtual firmware EFI. Workstation 14 automatically passed battery status to the VM. The Windows 10 host did not have display scaling (i.e. 100%). The VM resolution was set at 1920x1200 without scaling.

It was not a scientific test nor very controlled (I don't have timers, alarms that trigger the test; or initiate glxgears after 5 minutes screen goes blank, the glxgears did not run for the same duration). I simply boot up the VM but I let the Surface Pro 3 screen go blank (in essence to let it cool down, fans stop whirring) and then I do the glxgears.

You can see (in the attached CSV file) that for both OpenGL and DX11, it just degrades over time; which indicate some sort of throttling of the CPU (in turn the graphics chip embedded within the same package). The louder the fan got the slower the rate.

If your system is a laptop, you may want to set it up for "High Performance" for the Windows 10 host although Windows updates can make it disappear and you have to fiddle around with the registry to bring it back. Because with the default "Balanced", it will throttle many other things. And if the battery status is also passed to the VM, Xubuntu will also see it has a battery (although it is virtual) but the power states would be visible.

Just a (radical) thought, if you want to have Xubuntu as your daily machine to use, why not install Xubuntu as the host instead? Obviously there are other considerations and the problems of vmmon and vmnet breaking for Workstation Pro when there are VMware-unsupported kernel updates can be painful and counterproductive. Alternatively, you could just install Xubuntu as a physical machine by booting from an external USB HD/SSD. Most systems these days support external USB HDD/SSD boot that negates the need for complicated multiboot partitions; although Windows 10 will not proceed to install if it detects the target drive is an external USB drive.

[reijin]

Interesting, so your test basically confirms my results of the ~1200FPS ceiling. I'll check again for all the energy options, but I'm pretty sure I've set my Laptop to the maximum as this was my first idea how this came to be.

However, it still seems weird to me that enabling 3D acceleration always lowers the performance for me. Even in your case an improvement is visible. I noticed, I set up my machine using legacy BIOS instead of EFI... Could this have such a dramatic effect?

Regarding your thought: It is no option for me, as this setup is part of a security concept that I use to minimize the chance of compromise. Linux is my daily for security and working purposes and my Windows VM is only used for Windows related stuff (e.g. Office). The host is used basically never or only for non-internet related stuff in order to lower the chance of a compromise of my machine. Besides, I have some bad experiences with Linux running directly on Laptop hardware - it just had too many annoyances for me (suspending, drivers, power management and so on).