VMware Cloud Community
WoKeN
Contributor
Contributor

Can I share 1 Nvidia Quadro 6000 among multiple VMs?

Hello all, I am currently deciding the last piece of hardware I need to complete my rig to start building it.  Please help!

I need a video card capable of be utilized in different vms/guests simultaneously.  I was thinking about a Nvidia Quadro 6000 ever since I saw this video and the description for this video:

http://youtu.be/ME3xaLUTZgU?list=U

Tim Federwitz

1 year ago

I will have to answer this in multiple parts because of character limitation per comment. We are giving each VM 512MB of video memory. This reserves 256MB on the GPU and 256MB on the Host in RAM (so this will need to be taken into consideration in host sizing). The video card we are using is an Nvidia Quadro 6000 which has 6GB of VRAM. So in theory, and depending on the application and it's GPU load, you could assign 24 VMs to the card with 512MB video memory each.

I would just like to confirm with the VMware community if it's indeed possible to assign about 24 vms with this parameter of 512 mb of video memory per vm

And if this is actually possible, my last question would be how do I start and go about doing this?

Please help a poor fellow who has bought all the parts to his rig for the past 3 months and has been holding off from starting to put it together because of graphics card issue Smiley Sad

Reply
0 Kudos
10 Replies
admin
Immortal
Immortal

Hi WoKeN, short answer for your question is "Yes, you can share GPU (Nvidia Quadro 6000) among multiple VMs."

Hardware accelerated 3D graphics for Virtual Machines (VMs) was introduced with vSphere (ESXi) 5.1 release, and VMs on Hardware Version 9 (and later) can be configured to use hardware 3D acceleration. The vCenter Server interface provides for this configuration. vSphere 5.5 also is released now and adds support for more GPUs including certain AMD ones.

Not all guest operating systems will support 3D acceleration though. Only Windows 7 and Windows 8 will work as they get the SVGA 3D driver from VMware Tools. For Linux guest OS some of the newer releases of Ubuntu (Ubuntu 12.04 or later) and Fedora have "vmwgfx" driver (the 3D driver for Xorg).

ESXi 5.1 driver download:

NVIDIA DRIVERS VMware vSphere ESXi 5.1 Driver

ESXi 5.5 driver download:

NVIDIA DRIVERS VMware vSphere ESXi 5.5 Driver

Once you have sorted out the hardware compatibility (GPU with server) and ESXi 5.1/5.5 running, install the driver and allow GPU sharing as below:

1. Put server in maintenance mode:

  • vim-cmd hostsvc/maintenance_mode_enter

2. Perform driver installation, first we need extract the *bundle* file from the download zip. Once we have the bundle.zip file, transfer it to the ESXi host:

  • localcli software sources vib list -d <path-to-bundle>/NVIDIA-VMware-x86_64-319.65-1OEM.550.0.0.1331820-bundle.zip -n NVIDIA-VMware_ESXi_5.5_Host_Driver
  • vim-cmd hostsvc/maintenance_mode_exit
  • kill -HUP $(cat /var/run/vmware/vmkdevmgr.pid)   <== this is not required on ESXi 5.1
  • /etc/init.d/xorg start

Above commands are for host running ESXi 5.5, for ESXi 5.1 replace "localcli" command with "esxcli". At this point you should be able to power on 3D VMs. Below commands will help you to make sure you are on right track (in case above steps don't work).

Confirm hardware GPU is detected:

  • lspci | grep -i nvidia

Confirm Xorg started:

  • /etc/init.d/xorg status

Confirm Nvidia module got loaded:

  • vmkload_mod -l | grep -i nvidia

Create VM with Windows 7 or Windows 8 as guest OS (for example), before completing VM creation change Graphics configuration to use "Hardware" acceleration. This can be left to "Automatic" if you may have to move the VM at some point in future to another ESX host which may not have hardware GPU (in which case it will use software 3D rendering). Once you have the VM and guest OS setup, install VMware Tools. This gives VM the SVGA 3D driver. Reboot after tools installation and then your guest OS should be ready with hardware accelerated 3D. You can try some basic 3D benchmarks like 3DMark05 or so.

To confirm that the VM we created above is indeed using the graphic resources from the GPU (Quadro 6000), run the command "gpuvm" on the ESXi host, this will show the VMs which are using graphics memory. In my memory, I think the VM consumes the graphics memory it has been allocated entirely from the GPU. However there could be some overhead on the host memory, but I am not too sure about that.

Useful link with more information:

http://www.vmware.com/files/pdf/techpaper/vmware-horizon-view-graphics-acceleration-deployment.pdf

Hope this helps. Let me know if you have any questions.

Reply
0 Kudos
Linjo
Leadership
Leadership

Sayshh summarizes the answer very good but a few things I would like to add:

1. Be sure that your host and the GPU is supported, if you are fine to go without support then your options are somewhat broader.

2. With vSGA you will be limited to what OpenGL (2.1) and DirectX3D (9.0C) versions you will be able to use.

3. You do not write how you want to access the virtual machines and what issues you are trying to solve, for example watching videos will not benefit much from using GPU:s and not all remoting protocols can utilize them.

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
WoKeN
Contributor
Contributor

Wow thank you both so much for your very quick replies and very informative help!  I am very grateful.  Since the Quadro 6000 is basically a gtx 480 or little worse in performance I was wondering if it would be better to save about 400 dollars and buy a 300 dollar card that should perform better than a gtx 480 equivalent (quadro 6000)

So I was wondering if there is anyone out there who had success using a r9 280x or gtx 770 graphics card for svga in 5.5?  I'm trying to decide which of these two to buy

Reply
0 Kudos
Michelle50
Contributor
Contributor

Hi

Welcome to communities.

I think you should test your self before purchasing , as same device is working for some one not for another .

Reply
0 Kudos
WoKeN
Contributor
Contributor

You mean like buy the card and see if it works and then return if it doesn't?

Reply
0 Kudos
admin
Immortal
Immortal

The download links on Nvidia page show the product compatibility. The drivers available for ESX seem to (as published) support Quadro 4000, 5000, 6000 and Grid K1 and K2 cards. With other cards it could be hit-or-miss afair.

Reply
0 Kudos
Linjo
Leadership
Leadership

Other cards will not work, the driver checks the PCI-ID and will not load unless the card is one of the above.

There are ways of changing the PCI-ID on a GTX-card to a GRID-card and should be easy to find with some google skills. (Keywords: Nvidia mod k2)

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
WoKeN
Contributor
Contributor

Last year a thread was made which I only saw a couple hours ago:  Re: Radeon/Nvidia VMDirectPath

A good amount of those people are saying their AMD Radeon 7900 series or lower are working, pretty much anything up til a R9 290, one guy mentions his r9 270x works since it's a rebranded 7950 or something along those lines,  so would a r9 280x work too?  Does anyone have one and tried?

Reply
0 Kudos
Linjo
Leadership
Leadership

No, that thread is about passthrough one GPU to one virtual machine, not sharing one GPU between several virtual machines.

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
WoKeN
Contributor
Contributor

Thanks very much Linjo!  I missed that.  It's very unsettling to know I would spend at the least 600 dollars on a Nvidia Quadro 6000 seeing as it's just a tad slower than a gtx 480.  And I've read that upon gaming applications which I hope to use these vms for that a more up to date consumer card if it worked would operate better.  And since there are many consumer grade cards that are much faster than the gtx 480 such as the gtx 770 or r9 280 costs about 300 USD, it's a pity that a Quadro 6000 is my only reasonable option for price and performance

Reply
0 Kudos