Highlighted
Contributor
Contributor

Nvidia Quadro <3 vSphere 5.1?

Jump to solution

I assume that many of you with exictement have read through all news in vSphere 5.1

What cought my eye in View point of view was this:

"

Improved 3D Graphics Support – (View Only) –  hardware acceleration with possiblity to leverage NVIDIA’s hardware  cards installed in ESXi server, where those graphics cards are  virtualized and used in View Desktops. It’s targeted for graphics  intensive workloads, CAD designers, medical imaging etc…

NVIDIA Quadro 4000/5000/6000 and NVIDIA Tesla M2070Q are supported graphics cards.  Note that the ESXi Image profile must be installed with the NVIDIA GPU VIB file."

It certainly raises some questions for me:

a) The Nvidia GPU VIB-files is that free of charge or is it licensed from Nvidia or Vmware?

b) Have anyone tested 5.1 with Quadro-cards (and VMware Techie out there?) Pros? Cons?

c) Is the GPU fully accessible from the View guest or is there some kind of abstraction layer in between? After some research it seems like the abstraction layer is utilizing Xorg in a clever way.

202 Replies
Highlighted
User Moderator
User Moderator

Svga and vSGA has nothing todo with each other and you will not be able to tell from inside the VM that its using a GPU except for much improved 3D.

One tool that I use to verify is this one:

http://www.ozone3d.net/gpu_caps_viewer/

Run the OpenGL 2.1 benchmark and 600x600 and you should get about 30fps, without vSGA you would only get 1-3fps.

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
Highlighted
Contributor
Contributor

Goodmorning guys, hope u had a good weekend.

Week has started again. i just ran some of your commands to see what's installed and whats not.

here's the update:

esxcli software vib list | grep nvidia says nvidia vmware esxi 5.1 host driver vmware accepted

Esxcli system module load –module nvidia

unable to load module,  status : busy. seems to  me its loaded? correct me if im wrong ofcourse.

Esxcli hardware pci list –c 0x300 –m 0xff

I both see my quadro 600 and my quadro 4000 if i do that command.

when i do the nvidia smi command it gives me my quadro 4000 status, temps and gpu fan usage.

display 4MB / off 2047 mb

xorg service is still running.

same this happens when i try to boot up the VM using the hardware option,

device videocard is not supported by host!

and yes it is running on the right host!

Thanks in advance,


drxz

0 Kudos
Highlighted
User Moderator
User Moderator

Could you post the VMX-file and the vmware.log from the VM?

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
Highlighted
Contributor
Contributor

Hi,

Thanks for the update Smiley Happy

Kindly see the table below

~ # nvidia-smi
Mon Mar 25 09:44:56 2013
+------------------------------------------------------+
| NVIDIA-SMI 4.304.76   Driver Version: 304.76         |
|-------------------------------+----------------------+----------------------+
| GPU  Name                     | Bus-Id        Disp.  | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap| Memory-Usage         | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Quadro 4000              | 0000:05:00.0     Off |                  N/A |
| 36%   82C    P1    N/A /  N/A |  29%  594MB / 2047MB |      7%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Quadro 4000              | 0000:42:00.0     Off |                  N/A |
| 36%   52C   P12    N/A /  N/A |  15%  306MB / 2047MB |     12%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Compute processes:                                               GPU Memory |
|  GPU       PID  Process name                                     Usage      |
|=============================================================================|
|  No running compute processes found                                         |
+-----------------------------------------------------------------------------+

GPU utilization is is just 15% to 30%. How can I utiliz to its max.

0 Kudos
Highlighted
Contributor
Contributor
.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "9"
pciBridge0.present = "true"
pciBridge4.present = "true"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "true"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "true"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "true"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "true"
nvram = "win7hardware1.nvram"
virtualHW.productCompatibility = "hosted"
powerType.powerOff = "soft"
powerType.powerOn = "hard"
powerType.suspend = "hard"
powerType.reset = "soft"
displayName = "win7hardware1"
extendedConfigFile = "win7hardware1.vmxf"
floppy0.present = "true"
scsi0.present = "true"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsisas1068"
scsi0.pciSlotNumber = "160"
memsize = "2048"
scsi0:0.present = "true"
scsi0:0.fileName = "win7hardware1.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
sched.scsi0:0.shares = "normal"
sched.scsi0:0.throughputCap = "off"
ide0:0.present = "true"
ide0:0.fileName = "/vmfs/volumes/51236991-66a85ce8-fcd5-001517126726/en_office_enterprise_2007_DVD_VL_X12-19574.iso"
ide0:0.deviceType = "cdrom-image"
floppy0.startConnected = "false"
floppy0.fileName = ""
floppy0.clientDevice = "true"
ethernet0.present = "true"
ethernet0.pciSlotNumber = "192"
ethernet0.virtualDev = "vmxnet3"
ethernet0.networkName = "VM Network"
ethernet0.addressType = "vpx"
ethernet0.generatedAddress = "00:50:56:85:6b:7c"
usb.present = "true"
ehci.present = "true"
usb.pciSlotNumber = "33"
ehci.pciSlotNumber = "34"
svga.vramSize = "100663296"
vmci0.pciSlotNumber = "32"
guestOS = "windows7-64"
uuid.bios = "42 05 18 e1 22 13 a0 80-76 89 c0 a4 70 e2 50 72"
vc.uuid = "50 05 a1 e5 1b d5 a2 88-aa 93 25 48 ee 73 cd a9"
virtualHW.scheduledUpgrade.state = "none"
virtualHW.scheduledUpgrade.when = "never"
snapshot.action = "keep"
sched.cpu.min = "0"
sched.cpu.units = "mhz"
sched.cpu.shares = "normal"
sched.cpu.latencySensitivity = "normal"
sched.mem.min = "0"
sched.mem.shares = "normal"
tools.upgrade.policy = "manual"
replay.supported = "false"
replay.filename = ""
scsi0:0.redo = ""
pciBridge0.pciSlotNumber = "17"
pciBridge4.pciSlotNumber = "21"
pciBridge5.pciSlotNumber = "22"
pciBridge6.pciSlotNumber = "23"
pciBridge7.pciSlotNumber = "24"
scsi0.sasWWID = "50 05 05 6e a7 d8 29 b0"
hostCPUID.0 = "0000000d756e65476c65746e49656e69"
hostCPUID.1 = "000206d70020080017bee3ffbfebfbff"
hostCPUID.80000001 = "0000000000000000000000012c100800"
guestCPUID.0 = "0000000d756e65476c65746e49656e69"
guestCPUID.1 = "000206d700010800969822030fabfbff"
guestCPUID.80000001 = "00000000000000000000000128100800"
userCPUID.0 = "0000000d756e65476c65746e49656e69"
userCPUID.1 = "000206d700200800169822030fabfbff"
userCPUID.80000001 = "00000000000000000000000128100800"
evcCompatibilityMode = "FALSE"
vmotion.checkpointFBSize = "8388608"
softPowerOff = "FALSE"
unity.wasCapable = "true"
tools.remindInstall = "TRUE"
usb:0.deviceType = "hid"
usb:0.parent = "-1"
usb:0.port = "0"
usb:0.present = "TRUE"
usb:1.deviceType = "hub"
usb:1.parent = "-1"
usb:1.port = "1"
usb:1.present = "TRUE"
usb:1.speed = "2"
vmotion.checkpointSVGASize = "11534336"
machine.id = "vdi.broker.dedicatedPool=1;vdi.broker.useSvi=0;vdi.broker.brokerPublicKey=MIHxMIGoBgcqhkjOOAQBMIGcAkEA/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFwIVAJYu3cw2nLqOuyYO5rahJtk0bjjFAkBnhHGyepz0TukaScUUfbGpqvJE8FpDTWSGkx0tFCcbnjUDC3H9c9oXkGmzLik1Yw4cIGI1TQ2iCmxBblC+eUykA0QAAkEA7z+0cM02pw00rGsvBvjxRp2ZzMYKVPnPywe9LRmSj/n6BU4ruB+Y9N1lHyIbaVZncnQGwe7oU2IaltICRlEqCg==;vdi.broker.brokers=srv003.marin.local ;vdi.broker.agentIdentity=agent/bff6ef19-a6b0-4efb-a9ea-757601ea164d;vdi.broker.itemDn=cn=bff6ef19-a6b0-4efb-a9ea-757601ea164d,ou=servers,dc=vdi,dc=vmware,dc=int;vdi.broker.agentPrivateKeyHash=1D+y7SFiFjVykr+Jh9pHxA==;vdi.broker.asyncSessionSeconds=150;vdi.broker.singleuse2=0;vdi.broker.brokerPublicKeyHash=mE3INKxE62LHyHj7ifNLfQ==;vdi.broker.msMode=ON;vdi.broker.agentPrivateKey=MIHGAgEAMIGoBgcqhkjOOAQBMIGcAkEA/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFwIVAJYu3cw2nLqOuyYO5rahJtk0bjjFAkBnhHGyepz0TukaScUUfbGpqvJE8FpDTWSGkx0tFCcbnjUDC3H9c9oXkGmzLik1Yw4cIGI1TQ2iCmxBblC+eUykBBYCFA4HzvAOI+8S9PXvBFruauEK5dyy;vdi.broker.disconnecttimeout=0;vdi.broker.singleuse=0;vdi.broker.useSysprep=0;vdi.broker.poolDn=cn=win7-hardware,ou=server groups,dc=vdi,dc=vmware,dc=int"
svga.maxWidth = "3840"
svga.maxHeight = "2400"
mks.enable3d = "true"
mks.use3dRenderer = "hardware"
svga.numDisplays = "2"
hpet0.present = "FALSE"

Here's the vmx log.  trying to find the vmware.log file. its all new to me!

0 Kudos
Highlighted
User Moderator
User Moderator

This thread is becoming a bit messy with a few different issues being discussed.

I would recommend to post a new thread if its not directly related to what have been previously discussed.

The GPU will not be used 100% unless there is enough work for it todo, same as any CPU...

I think it looks healthy with a range from 15% to 30% it means that you have some headroom and have not maxed out the CPU.

I am sure you can get it to 90%+ if you are running some GPU benchmark apps in the VM:s

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
Highlighted
Contributor
Contributor

Hi, do you know if vSGA works with nVIDIA's latest K4000 or K5000?
It seems only Fermi architecture is supported at the moment.
Not sure if there is any Kepler compatible ESXi driver available right now.

Any experience on that please?

One more question: would vMotion work on a VM that has video hardware acceleration enabled?
Cheers.

0 Kudos
Highlighted
User Moderator
User Moderator

Can't see anything that stands out in the vmx-file except for the size of the svga.vramsize, try to set it to 512Mb in the vSphere GUI and enable 3D.

The vmware.log is usually in the same folder as the vmx-file on the datastore.

Is this vm a newly built machine or has it been converted or used for something else?

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
Highlighted
Contributor
Contributor

it has been on 512 for the last few days to try. didnt help.

Its a standard template so there are more vm's that use the same template.

Will try to make a clean install and see what that does.

let me check again for the log file.

does it help if i configure the gpu for passtrough or isnt that neseccary?

Thanks.

edit,

Logfile included

0 Kudos
Highlighted
Enthusiast
Enthusiast

Hi,

Anyone tried with inferior cards, for example quadro 3000 or 2000?

0 Kudos
Highlighted
Enthusiast
Enthusiast

Unfortunately only the Quadro 4000-6000; VGX K1 and K2 (GRID), and a one other Tesla (cant remember the model) are currently supported with the 304.79 driver.  The Keplar Quadro cards (K4000/K5000) are not supported; hopefully that will only be a "yet".

0 Kudos
Highlighted
User Moderator
User Moderator

The GPU should not be marked for passthrough.

Could you post the output from the "gpuvm" cli-command?

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos
Highlighted
Contributor
Contributor

gpuvm command gives nothing.

just goes to the next input line without any feedback!

0 Kudos
Highlighted
Enthusiast
Enthusiast

"would vMotion work on a VM that has video hardware acceleration enabled?"

     Yes vMotion will work with vSGA enabled VMs.  Also if you are going between a vSGA enabled host and a non-vSGA enabled host if you have 3D rendering set to "Automatic" then it will also support vMotion to hosts without GPUs using Soft3D.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Is your xorg service running?

issue "/etc/init.d/xorg status" to check.  If it is not start it.

0 Kudos
Highlighted
Contributor
Contributor

xorg is running yes!

allthough i started it in vsphere, where it says running. when i entered the command it said not running.

started it,  but still same problem that the vm wont start with the same device videocard is not supported.

0 Kudos
Highlighted
Contributor
Contributor

okay, now i do get input from the gpuvm command.

xserver unix:0, gpu maximum memory 2076672KB

gpu memory left 2076672kb

thats it.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Have you tried to start the vm again now that "gpuvm" gives you some output?  Thats basically just saying how much GPU VRAM is avaliable to divy out to vSGA enabled VMs, which currently you dont have any powered on (or unable too in your case).

0 Kudos
Highlighted
Contributor
Contributor

It seems to be working guys,

10 vm's working on this gpu now. 5 of them are now playing a 1080p movie without a problem.

I dont know how to thank you guys, but really thanks!

also got our teradici server offload card working.

Thanks, have a great week.


DrxzEEh

0 Kudos
Highlighted
User Moderator
User Moderator

Great to hear! Happy it now works for you.

Please give us feedback on how you think it performs and the usecases that you see.

// Linjo

Best regards, Linjo Please follow me on twitter: @viewgeek If you find this information useful, please award points for "correct" or "helpful".
0 Kudos