I assume that many of you with exictement have read through all news in vSphere 5.1
What cought my eye in View point of view was this:
"
Improved 3D Graphics Support – (View Only) – hardware acceleration with possiblity to leverage NVIDIA’s hardware cards installed in ESXi server, where those graphics cards are virtualized and used in View Desktops. It’s targeted for graphics intensive workloads, CAD designers, medical imaging etc…
NVIDIA Quadro 4000/5000/6000 and NVIDIA Tesla M2070Q are supported graphics cards. Note that the ESXi Image profile must be installed with the NVIDIA GPU VIB file."
It certainly raises some questions for me:
a) The Nvidia GPU VIB-files is that free of charge or is it licensed from Nvidia or Vmware?
b) Have anyone tested 5.1 with Quadro-cards (and VMware Techie out there?) Pros? Cons?
c) Is the GPU fully accessible from the View guest or is there some kind of abstraction layer in between? After some research it seems like the abstraction layer is utilizing Xorg in a clever way.
drxzeeh,
Good to hear it is working. By the way, do you know what wasnt working? How did you get it to start working? I'm curious to hear what the issue was.
Regards,
Nathan
Well, first of all that xorg service, i started it using the vsphere client but that didnt seem to cut it. after you gave me that init.d command it finally started.
after starting that the existing vm's didnt work. i made a new template with a clean windows 7 installation. ( also for the teradici client software )
that did seem to cut it. for the rest its pretty simple but just a few mistakes such as:
No dual gpu. ( got no gpu on chip )
the vib driver installation was fine, i used the update manager in vSphere
but the most important was the dual gpu, the xorg service and a clean template.
the cli commands Linjo gave me gave me more insight in what was working and what wasnt. really helped me in troubleshooting the problems.
For me it seems that starting the XORG service did it. I kept my VM (v9) that I had used and it cut the available pool by 256Mb.
Odd thing is I set the pool to use 512Mb. I did see my Windows performance index go from 3 to 6 under 3D performance. I will try 3D Mark next for a more accurate reading.
Anyone have any thoughts as to why only 256Mb is being reserved? Also, anyone else get this to work on a card besides NVIDIA GRID i.e. maybe we can create a "it works" list of video cards.
(Full disclosure - I am running this on a modified GTX 680 into a GRID K2 with 1x GPU and 2Gb of vid RAM soooo.........it's a little MacGyver)
That is interesting about your windows performance index being a "6" with an emulated K2. I'm using a Quadro 4000 and have the same WPI. I was thinking that if I move to a more capable card (more gpu cores) that score may go up some.
I was wondering if anyone else found that nvidia modding thread and was going to post results here. I came across it two weeks ago, but havent gotten a chance to play with a modded 670 or 680 yet.
Technically speaking any Geforce/Quadro/Tesla card will work. It is only restricted by nVidia's current driver to only load when it sees a "supported" device ID. If you fool the driver into seeing what it thinks to be a supported device ID from its list it will work. I was told that the very first driver nVidia created for vSGA worked with GeForce cards as well, but then once they released an update it restricted support for only pro level cards.
Let us know what kind of 3D Mark 06 score you get from the first test. I could only get the first two tests to run. It would always through Direct3D device lost errors and quit before it could run any of the other tests. The highest score I've gotten with the first test was 4200 frames in the alotted test time. That was with the software view client.
Regards,
Nathan
Well I got accidental damage protection for the 680 so I figured "why not". I started with that modding thread and eventually arrived here once I couldn't figure out the VMware side.
[soap box] I really wish NVIDIA had supported other lower end cards with their drivers natively. The GRID cards cost $2k+ and that's really hard to choke down when you're just tinkering around at home. $500 I can swing. The price of a used Honda, I can't swing. [end soap box]
They are modding new cards all the time on that forum so some cheaper cards should be available soon. The 680 was just the easiest to mod (until the find the mod for the second GPU on the 690).
With the 4000 are you doing a shared GPU or pass-through? NVIDIA's site lists a few other cards than the K2 included in that driver but I didn't know if those other cards supported virtualization.
I'm using the Quadro 4000 with vSGA; not doing passthrough at the moment. Yes the other cards it lists are all supported with shared graphics (vSGA) Quadro 4000/5000/6000, Tesla M2070Q, and GRID K1/K2.
If you didint know already there is also a reg setting that opens up the frame rate lock on the devTap driver. By default it is locked to 30 FPS. To change it add this reg setting..
[HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware SVGA DevTap]
"MaxAppFrameRate"=dword:00000000
This is only useful if you are trying the get the maximum performance out of a certain application and not too concerned with sharing out the GPU resources efficiently to all your other VMs. The VM that has this setting will use more GPU resources than the other VMs that don't have this setting. This was put in place to set a cap on how much GPU resources a particular VM could use.
In theory 5.2 is supposed to remove the need for the registry tweak. Although I am guessing that means when a view session is established.
Here is some odd findings re 3DMark 06 v1.02.
When I run the benchmark through the console I can complete all of the tests and I get a 3DMark score of 11580 w/ the registry key set (it was like 7580 w/o the reg key).
When I try to run the benchmark through a view session, I get the same device lost error. I'm guessing it is due to a crappy video drive by VMware.
The registry tweak that 5.2 takes care of is not the devtap frame rate lock. The reg tweak that you are talking about is a network tweak, which modifies the FastSendDatagramThreshold value. It can be found here - HKLM\System\CurrentControlSet\Services\Afd\Parameters
FastSendDatagramThreshold (set it to 1500).
What is your 3DMark score for the first test? I was able to get past the initial errors when using the view client by hitting "ok" on the error dialog box and trying to run the benchmark again. The first test was the only one that would "eventually" run. Yes, I'm not sure what is going on with the view client not being able to run through the entire set of tests. Would be nice if it would.
Google Earth will also not show 3D at all when using DirectX. You have to move over to OpenGL mode to even get 3D objects to show and even then the colors are off (they are on some type of grey scale only... no colors).
The probable reason for the the device lost is that 3D Mark is changing resolution quite rapidly and PCoIP can't keep up with or might not support that resolution.
// Linjo
I got my Quadro 4000 installed and the Nivida driver installed. But I can´t start the xorg service that everyone is talking about. It jus says "Xorg is not running" I tried the "/etc/init.d/xorg start" command but same result. After reading this thread I put the console display at the onborad GPU. Same result.
I have no vm running on the host. Can anyone give me a clue what to do?
when i run the "nvidia-smi" command it returns:
Failed to initialize NVML: Unknown Error
No problem, we will figure this out.
Could you post the output from these commands?
gpuvm
Show what VMs are using gpu(s)
Esxcli software vib install –v /path-to-vib/name-of-vib.vib
Loads the NVIDIA VIB
esxcli software vib list | grep NVIDIA
Verify NVIDIA vib is installed
Esxcli system module load –module nvidia
Verify NVIDIA module load
Esxcli hardware pci list –c 0x300 –m 0xff
Verify devices are present
nvidia-smi
General status of the GPU / driver version
// Linjo
Thanks for quick respons. Dont you never sleep?
gpuvm -> no result
Esxcli software vib install –v /path-to-vib/name-of-vib.vib ->
Installation Result
Message: Host is not changed.
Reboot Required: false
VIBs Installed:
VIBs Removed:
VIBs Skipped: NVIDIA_bootbank_NVIDIA-VMware_ESXi_5.1_Host_Driver_304.76-1OEM.510.0.0.802205
esxcli software vib list | grep NVIDIA ->
NVIDIA-VMware_ESXi_5.1_Host_Driver 304.76-1OEM.510.0.0.802205 NVIDIA VMwareAccepted 2013-04-03
Esxcli system module load –module nvidia
# esxcli system module load -m nvidia (module, does not work for me!?) ->
Unable to load module /usr/lib/vmware/vmkmod/nvidia: Busy
Esxcli hardware pci list –c 0x300 –m 0xff ->
000:004:00.0
Address: 000:004:00.0
Segment: 0x0000
Bus: 0x04
Slot: 0x00
Function: 0x00
VMkernel Name:
Vendor Name: Matrox Graphics, Inc.
Device Name: G200eR2
Configured Owner: Unknown
Current Owner: VMkernel
Vendor ID: 0x102b
Device ID: 0x0534
SubVendor ID: 0x1014
SubDevice ID: 0x0405
Device Class: 0x0300
Device Class Name: VGA compatible controller
Programming Interface: 0x00
Revision ID: 0x00
Interrupt Line: 0x0b
IRQ: 11
Interrupt Vector: 0x21
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x0221
Module ID: -1
Module Name: None
Chassis: 0
Physical Slot: 0
Slot Description: Matrox G200
Passthru Capable: true
Parent Device: PCI 0:3:0:0
Dependent Device: PCI 0:3:0:0
Reset Method: Bridge reset
FPT Sharable: true
000:08b:00.0
Address: 000:08b:00.0
Segment: 0x0000
Bus: 0x8b
Slot: 0x00
Function: 0x00
VMkernel Name:
Vendor Name: NVIDIA Corporation
Device Name: NVIDIAQuadro 4000
Configured Owner: Unknown
Current Owner: VMkernel
Vendor ID: 0x10de
Device ID: 0x06dd
SubVendor ID: 0x10de
SubDevice ID: 0x0780
Device Class: 0x0300
Device Class Name: VGA compatible controller
Programming Interface: 0x00
Revision ID: 0xa3
Interrupt Line: 0xff
IRQ: 255
Interrupt Vector: 0x61
PCI Pin: 0x70
Spawned Bus: 0x00
Flags: 0x0201
Module ID: -1
Module Name: None
Chassis: 0
Physical Slot: 5
Slot Description: PCI-Express Slot 5
Passthru Capable: true
Parent Device: PCI 0:128:2:2
Dependent Device: PCI 0:128:2:2
Reset Method: Bridge reset
FPT Sharable: true
nvidia-smi ->
Failed to initialize NVML: Unknown Error
Is the Quadro 4000 configured for pass-through? If it is, unconfigure it for pass-through.
Is the onboard video adapter configured as the primary output device in the BIOS? If it is not, set it as primary output device.
Tjek!
Pass-through is not enabled and the active display is the onboard gpu device.
The host is running ESXi 5.1.0, 799733. IBM 3650 M4. Vcenter: 5.1.0 7999731 could that some how be a issue?
I will make sure that I have the lastest firmware..brb!
If you are using View 5.2 (which supports graphics virtualization) you should be on at least ESXi build 838463 (5.1.0a).
It is now upgraded to ESXi build 838463. I still cant start xorg with this command "/etc/init.d/xorg start"
gpuvm -> still no result
Esxcli software vib install –v /path-to-vib/name-of-vib.vib ->
Installation Result
Message: Host is not changed.
Reboot Required: false
VIBs Installed:
VIBs Removed:
VIBs Skipped: NVIDIA_bootbank_NVIDIA-VMware_ESXi_5.1_Host_Driver_304.76-1OEM.510.0.0.802205
esxcli software vib list | grep NVIDIA->
NVIDIA-VMware_ESXi_5.1_Host_Driver 304.76-1OEM.510.0.0.802205 NVIDIA VMwareAccepted 2013-04-04
Esxcli system module load –module nvidia -> "esxcli system module load –m nvidia" i use "-m" instead
Unable to load module /usr/lib/vmware/vmkmod/nvidia: Busy
Esxcli hardware pci list –c 0x300 –m 0xff ->
000:004:00.0
Address: 000:004:00.0
Segment: 0x0000
Bus: 0x04
Slot: 0x00
Function: 0x00
VMkernel Name:
Vendor Name: Matrox Graphics, Inc.
Device Name: G200eR2
Configured Owner: Unknown
Current Owner: VMkernel
Vendor ID: 0x102b
Device ID: 0x0534
SubVendor ID: 0x1014
SubDevice ID: 0x0405
Device Class: 0x0300
Device Class Name: VGA compatible controller
Programming Interface: 0x00
Revision ID: 0x00
Interrupt Line: 0x0b
IRQ: 11
Interrupt Vector: 0x21
PCI Pin: 0x78
Spawned Bus: 0x00
Flags: 0x0221
Module ID: -1
Module Name: None
Chassis: 0
Physical Slot: 0
Slot Description: Matrox G200
Passthru Capable: true
Parent Device: PCI 0:3:0:0
Dependent Device: PCI 0:3:0:0
Reset Method: Bridge reset
FPT Sharable: true
000:08b:00.0
Address: 000:08b:00.0
Segment: 0x0000
Bus: 0x8b
Slot: 0x00
Function: 0x00
VMkernel Name:
Vendor Name: NVIDIA Corporation
Device Name: NVIDIAQuadro 4000
Configured Owner: Unknown
Current Owner: VMkernel
Vendor ID: 0x10de
Device ID: 0x06dd
SubVendor ID: 0x10de
SubDevice ID: 0x0780
Device Class: 0x0300
Device Class Name: VGA compatible controller
Programming Interface: 0x00
Revision ID: 0xa3
Interrupt Line: 0xff
IRQ: 255
Interrupt Vector: 0x61
PCI Pin: 0x28
Spawned Bus: 0x00
Flags: 0x0201
Module ID: -1
Module Name: None
Chassis: 0
Physical Slot: 5
Slot Description: PCI-Express Slot 5
Passthru Capable: true
Parent Device: PCI 0:128:2:2
Dependent Device: PCI 0:128:2:2
Reset Method: Bridge reset
FPT Sharable: true
nvidia-smi->
Failed to initialize NVML: Unknown Error
conclusion = no differents. I really can´t figure it out.
Try to start it from vSphere. It took about 5 minutes for mine to start up.
Also check to make sure you put the VIB in as an extension and not a patch.
What do you mean by "put the VIB in as an extension and not a patch"?
If you used UpdateManager you can create a baseline to install the nvidia drivers (.zip file). You want to make sure you select the option of extension and not patch when asked.
With XORG not starting that usually is a driver installation problem (maybe indicated by the "skipped" category?).