WMware Player GPU Utilization Cap?

[SuperRoboWizard]

For some reason, the guest seems to only be able to use 1/4 of the total GPU. (There's also a 2GB VRAM cap, but that isn't as big of a problem). I'm really not sure what's going on here... the guest reports 100% utilization, while the host only shows 30% and the other 5% is probably the host (ubuntu 16.08). Do I have the virtual machine configured wrong, or do I have to edit a config, or what?

Using: GTX 960, nvidia-smi, rivatuner + MSI afterburner

(Not sure why it says D3D11, it clearly isn't, but whatever)

[wila]

Hi,

The virtual machine does not get direct access to the GPU, it only sees a virtualized graphics card.

--

Wil

[SuperRoboWizard]

Yes, I do understand that, as far as Windows knows, it is fully using the GPU. But from what I am seeing, VMware Player has the virtualized video card only represent a small portion of the real card. Shouldn't the virtual card be able to completely use all available resources on my real GPU?

[wila]

Hello,

Eh no, unfortunately that is not how it works.

Here's an excerpt from this article: Frequently Asked Questions about VMware Fusion

(I realize that you are not using Fusion, but the concepts are the same, just read "Player" whenever you see Fusion and "PC" when you see Mac)

Virtual Hardware

Description

Virtual machines see a very different set of virtual hardware than is  actually on the host. The most commonly asked-about one is the video  card; other examples include (but are not limited to) the network card,  keyboard/mouse, drives, and more.

A related question is why a device (such as an optical drive or a USB device) can only be used by one OS at a time.

A related question is why you can't dedicate a PCI cards to a guest.

Explanation

One of the key concepts of virtualization is resource control (along with equivalence and efficiency). In other  words, a guest should not be able to affect things that the  virtualization software does not allow it to affect. This was one of the  major challenges to x86 virtualization - there are certain x86  instructions that cannot be easily handled, and was why VMware's Binary Translation technique was a big deal when it was new - it made x86  virtualization possible.

Many, if not all, devices assume they are controlled by exactly one OS -  that is, whoever is talking to them is the one they should listen to.  If two or more OSes were to give conflicting commands, devices would get  confused, and then so would the OSes as they started to get unexpected  errors. In these cases, we must either dedicate, or passthrough, the device to one OS, or emulate a similar device. Passthrough devices must be safe in the sense that  anything a guest can do to a passed-though device must maintain the resource control criteria. As a concrete example, passthrough devices  include USB devices, emulated devices include the default keyboard/mouse  and sound.

Note that even Fusion goes through the host's drivers (again, only one OS controls the hardware, so that means everything funnels through the host drivers) - this means we're subject to any bugs or limitations of the host drivers.

Video cards are an example of a device which assumes it is controlled by  exactly one OS. If a guest were to be able to access a graphics card  directly, it could draw anywhere on screen it wanted, affect host  textures, etc. Even a well-intentioned guest would cause problems,  because it wouldn't be aware of what the host is doing ("Hey, what's this texture? I don't recognize it, must not be important!" and then  your windows/icon/desktop/menus/etc. disappear). It's not possible to  dedicate an entire graphics card to the guest either, since the  underlying buses are also not safe to pass through - see for example Re: Guest able to directly access PCI cards for a good explanation. Even if you did this, the only Mac which could  take advantage of this would be a Mac Pro with extra graphics cards.

Because of this, we take the emulation approach. The guest sees a VMware  video card, and we do the work of converting guest commands into  something that's safe and usable for the host video card. There's no  point in installing drivers for the host video card in the guest (with  the exception of Boot Camp virtual machines, where you might want to  native boot) since the guest never gets to speak directly to the host video card.

One future possibility is the notion of virtualization-aware hardware, which does not make the assumption that it's only ever talking to one OS. Such  hardware would have different contexts that the host can switch between  for its own use or for guest use. Intel's Vanderpool and AMD's Pacifica  are examples of virtualization-aware CPU technology. Other  virtualization-aware hardware, such as for graphics, network, or  storage, is theoretically possible but I don't think any currently  exist, especially not for the consumer market. I'm am not sure when or  even if they might become available.

So there's a translation layer that takes care of the virtual graphics. That translation layer is not ideal,  for example the highest supported DirectX in your guest is DirectX 10 and OpenGL 3.3. It takes VMware some time to add more advanced support as writing that translation layer is very complex.

See also: http://www.vmware.com/products/workstation.html#compare

Also note that there currently is no PCI pass--through support for Workstation/Player or Fusion so no way to directly access a GPU from within a PC.

--

Wil

[SuperRoboWizard]

I think there's a miscommunication problem here. I'm not sure if I'm not understanding what you're saying, vice-versa, or both, but I'm going to try to explain my thought process more verbosely, and hopefully that will help.

So I made this thread thinking that the way the virtual GPU worked was this:

     -the virtual hardware gets any requests from the guest to do something with the video card

     -the VMware CPU process alters the video requests to use OGL instead of DX, and other things (like drawing to the window, not the whole screen, and altering positions in VRAM)

     -these altered commands are seen by the GPU just like any other 3D application

     -the video card takes the instructions and does it's thing

I initially thought that the problem here was that in step 3, for some reason if VMWare saw that this process was using too much of the host's compute, it would slow it down.

What I'm getting from the quote is that the problem lies in step 2 - this process of re-arranging and converting - is slow. Because things would get out of order if it was executed across multiple CPU cores, a glance at my CPU monitor seems to indicate that it is not the bottleneck, as the VMWare process is not using all four cores to their maximum potential. But this process means the GPU gets these draw requests far slower than it would otherwise, so it doesn't have to work very hard to keep up with them. Is this what's actually going on?

[wila]

Hi,

Not everything actually gets executed by the GPU even, part of the logic even runs on the CPU.

The VRAM you reference is mostly just RAM, but it is tagged as VRAM for your virtual GPU.

The virtual GPU has to abstract a lot. It's not just your GPU card it has to support, but all of the supported cards (and preferably a lot more as that)

So yes the translation layer cannot translate everything directly from virtual GPU to physical GPU the way you would want to have it translate as it has to be supported by the common drawing primitives that the abstraction layer understands.

Take also into account that you can have multiple virtual machines writing to the same GPU at the same time AND have the host write to it.

A scenario that you would see when you have multiple windowed guests running (eg. you would see multiple guests + your host OS drawing on the same "canvas")

So what you are seeing is pretty much expected.

--

Wil

[SuperRoboWizard]

With all due respect, I feel that you aren't entirely understanding the question.

The bottom GPU counter in the image is from the host, which means that is the actual use.

My question is:

    -The application on the guest thinks it is using 100% GPU

    -The real GPU is only 30% used

I understand that the guest can't have direct access to the GPU to prevent conflict, but through emulation, it should be able to use all available compute:

for example, let's say that the host is using 5% of the real GPU, and the guest is under full load (a benchmark, for example), equating to 100% use of the virtual GPU.

Shouldn't that result in 100% total usage on the real GPU, with 5% being used by the host, and 95% used by the graphics emulator?

It clearly isn't working this way, but why not?

The only thing in your response that I saw was addressing performance was with multiple guests, which is not possible with VMware Player, and should result in lower framerates for each guest, but still should have the real GPU running at 100%.

[wila]

Hi,

With the same respect I do think I understand your question, it's just not working as you expect it to work.

It can not fully utilize your GPU because it never accesses the GPU directly, the application on the guest never gets to see the real GPU, so whatever it thinks the GPU is doing is what the virtual hardware tells it. The virtual hardware does not expose all of your physical GPU, so it cannot fully utilize that.

On top of that there are other bottle necks such as the extra dependency on functionality emulated on the CPU and the VRAM the guest thinks it has actually being RAM.

--

Wil

[SuperRoboWizard]

So, in theory, let's say I upgrade my system.

Just for the sake of argument, if I replaced my 960 with a Titan XP, would that improve performance?

What about replacing my 4690K with a 6950X?

Or upgrading to 2133 RAM?

Is this an issue where the virtual machine can only use a certain portion of the real system, or is there effectively a 'wall' for the upper end of performance, limited by the inner workings of the virtual machine? Or am I being bottlenecked by something like my SSD or RAM, which would lower my framerate without making any performance counters be pinned to 100%?

[wila · Accepted solution]

Hi,

While there might be small improvements, I don't think it will make much of a difference on any of the upgrades you are suggesting and you'll just end up spending a lot of money on no or barely noticeable differences.

If you want a very performant GPU then virtualisation isn't the best match I'm afraid due to how it works.

The only option you would have is to get a product that does PCI passthrough and allows you to directly present the graphics card to the virtual machine. (Beware that your host then has NO graphics card, unless you have more as one)

VMware Player does not offer PCI passthrough (nor does any of the other VMware Desktop products)

--

Wil

[SuperRoboWizard]

Ok, thanks. That's what I was getting at. I wasn't actually intending to buy any of those things, they were just extreme examples to help demonstrate my thought process.

So I guess I'll just be waiting for VMware to improve their virtual GPU, or get a second monitor, dedicating the VM to games and running the host off of my CPU's integrated graphics. I remember hearing about a program that would let you use the same mouse across different computers using the local network, so that would probably be the way to do it.

Do you know of any virtual machine software that would allow a GPU passthrough? I know about KVM, but it seems overly complicated for what I'm trying to do, and I'd much prefer a graphical editor.

[wila]

Hi,

Seems that virtual box has some support for it if you run from a linux host.

https://www.virtualbox.org/manual/ch09.html#pcipassthrough

and Hyper-V has it if you run it from a Windows Server OS

http://vinfrastructure.it/2015/11/device-passthrough-microsoft-hyper-v/

But I have no experience with either on this particular subject I'm afraid, so don't know how well it works.

--

Wil