How to use vGPU and vSGA both on same vSphere 6.0 host

[whibr]

I have been testing vGPU on a vSphere 6.0 host with Horizon 6.1, and I now want to add a vSGA pool to this host. Only two physical GPU's are being used with the two vGPU vm's I have powered on.

My question is, what do I need to do to make this host allow vSGA-enabled VM's to power on and share resources on the other two remaining GPU's?  I set the desktop pool to use "Hardware" 3D Renderer and added one VM to the pool. 

When I try to power on this VM, I get this error message in vSphere Web client: 

"Hardware GPU is required but not available. The virtual machine will not start until GPU resources are available or the virtual machine is configured to allow software rendering."

This is the gpuvm and nvidia-smi command line output:

The ESXi 6.0 host has two GRID K2 cards for a total of 4 GPU's. Only two vGPU-enabled vm's are powered on, so it would seem I can use the other two GPU's for vSGA VM's, right?

[Linjo · Accepted solution]

That is currently not possible since the Nvidia VIB:s for vSGA and vGPU can not be loaded at the same time.

The driver will claim all compatible GPU:s (except for the ones configured for passthrough, IE vDGA)

One exception to this is if you have one AMD GPU for vSGA and one Nvidia GPU for vGPU.

Cannot comment on if and when this will be solved, maybe krd can do that.

// Linjo

[krd]

Linjo's response is correct, a single ESX systems cannot support VMware vSGA and NVIDIA vGPU simultaneously.  This is because these graphics features have separate VIBs that cannot coexist on same system.

VMware and NVIDIA are aware of this restriction and are working to resolve it in a future release.

[whibr]

Linjo, krd,

Excellent, thank you both for the clarification..one follow up question then.  I have another vSphere 5.5 host with NVIDIA GRID K2's, currently hosting several vSGA pools and two vDGA vm's.

So, I would like to upgrade this system to vSphere 6.0, which VIB will I need to install that does not enable vGPU?  I want to keep the vSGA and vDGA pool vm's running on this host intact after upgrading to ESXi 6.0.

Or would you recommend leaving this as a vSphere 5.5 until we are ready to migrate those pools to vGPU?

[krd]

Yes, you can maintain your current setup with vSphere 6.0.  Nvidia has vSGA and vGPU drivers for vSphere 6.0 available now. Here are current download links:

  NVIDIA DRIVERS VMware vSphere ESXi 6.0 Driver # vSGA

  NVIDIA DRIVERS NVIDIA GRID VGPU SOFTWARE RELEASE 346.42/347.52 WHQL # vGPU

Note NVIDIA periodically updates their drivers, so you should use their driver download link to find the latest.

[Linjo]

There are VIB:s provided by NVIDIA, the easiest way to tell them apart is the the vGPU one have "vgx" in the name.

They are both available from www.nvidia.com/drivers, choose "GRID" and you can choose either "GRID Series" (vSGA) or "NVIDIA GRID vGPU" (vGPU)

Then pick the GPU and then the vSphere version, for vGPU only vSphere 6 is supported.

There should not be any problems upgrading to vSphere 6 and continue using vSGA, but you will have to manually update the VIB.

// Linjo

[krd]

One minor note about upgrades.  When upgrading from vSphere 5 to 6, as Linjo said, you need to manually remove prior NVIDIA vib and install new one.  This is required because NVIDIA vSphere 5 vibs use some interfaces that are not compatible across releases.  VMware and NVIDIA have updated the vSphere 6 drivers to strictly use compatible interfaces, so subsequent upgrades will be easier because the current driver should continue to function.

[whibr]

Thanks again, fellas.  Our first vSphere 6.0 install was to upgrade our GRID K2 vSGA server from vSphere 5.5 using vCenter Server Update Manager.  After the initial upgrade scan of the host, it showed an error message regarding the currently installed VIB as being incompatible with vSphere 6.0 and that it needs to removed.