I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it. I actually had this working on a previous system but I had to reinstall. Now when I do this the guest fails to start and I get the following:
Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User
When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management. Any ideas on why this is occuring and why it worked at one point but now it fails?
I was also unlucky with nvidia cards. I noticed the same problems with a 8400 and a GT210.
For the WDDM, I did as you did; I installed the tools provided by my ESXi (4.1U1, I haven't done any other updates).
I'd be interested in more details about the configuration working with DXVA (ESX version, driver, ...). I didn't checked if I had VDDM 1 or 1.1 though.
Apologies in advance for the huge post...
Here are the details of my current setup:
Motherboard: SuperMicro X8SIA-F
Processor: Intel Xeon X3440
RAM: 16GB total (4x4GB) Registered ECC DDR3 (Kingston KVR1066D3Q8R7S/4G)
Add-on Cards:
ESXi version: 4.1.0, 260247
Virtual machines:
Suggestions for anyone encountering BSODs when booting a Win7 VM which uses a VT-d GPU:
Other suggestions:
Thanks for the most elaborating post somedude1234
Some things I learned from your post regarding software (as hardware is obviously different)
1.) Is there a way you can try to see if this setup works with the latest ESXi 4.1u1 which is version: 4.1.0, 348481?
I will try and test on the version you are using, maybe it the GPU passthrough got broken somehow on the newer release...
2.) The WDDM version of your tools is also lower then the lates, I will try that as well.
3.) Did you install the windows 7 with the GPU passthrough or only added it after the OS was installed?
Regarding hardware:
As I did try a PCI video for the ESXi, I still hope I will be luckier with a different combination, so I might search for a Matrox G200 PCI although I could only see G450 available (on ebay). The Radeon 6850 is still pretty expensive for a test purpose I would buy it in a second if I was completly sure it will work, I hate good hardware just dusting up in my pile of unusable good things....
A small tip I can give regarding your storage with Nexentastore, I also tried it first, but currently it doesn't support the vmxnet3s driver for the network which limit the network interface to 1gb which is 128mbytes, which is nice..... but....
I ended up installing OpenIndiana (solaris variant) with napp-it (the gui for the storage), that suppots the vmxnet3s 10gb card (from the vmware tools).
I was pleasently surprised to get on a virtual machine with its HD attached to an NFS storage to the OpenIndiana, I got about 430mbyes read and 270mbytes write speed, I have raidz2 with 6 drives but even with less drives the 1gb ethernet limit would have been reach sooner.
When Nexenta support this driver I will probably move to it as the GUI and CLI are much more impressive.
You're welcome!
1) I built the system back in the Dec/Jan timeframe and haven't updated the software since it was put into production. I've been contemplating an update to ESXi, I might give it a shot over the long holiday weekend.
2) Yes, I'm using the VMware tools version that is included with my slightly down-rev version of ESXi, which likely explains the VMware SVGA 3D driver also being slightly down-rev. I believe I misstated in an earlier post that you need WDDM 1.1 drivers for all of your mis-matched video cards in order to work correctly in Windows 7. I based this statement on the WDDM wiki entry, specifically, the following is listed as a "new feature" with WDDM 1.1:
- Support multiple drivers in a multi-adapter and multi-monitor setup
In my case, it appears that my AMD 6850 has a WDDM 1.1 driver, while the VMware SVGA 3D driver is WDDM 1.0.
3) I created the VM, installed Win7, ran windows updates, and then shut the VM down to add the GPU via PCIe passthrough.
Note that I borrowed an old AMD RadeonHD 3450 for my "proof of concept" testing before ordering the 6850. I was successful with both the 3450 as well as the 6850. I would expect that any of the ATI/AMD PCIe cards between the 3450 and the current 6800 series should work fine, but you never know unless you test a specific combination of hardware.
Thanks for the tip on OpenIndiana support for VMXnet3. I was quite disappointed when I realized that I woudln't be able to utilize this driver during my build (all of my other VM's are running with this driver, it's too bad the NAS is the one that doesn't work). At the time of my build, I considered OpenIndiana + NAPP-IT for the NAS, but I felt like OpenIndiana was a bit "too new" at the time for my data. You've inspired me to build an OpenIndiana VM and do some A/B testing with the same array to see how much of a difference it makes. In theory, I can just re-assign the SATA controller from the Nexenta VM over to the OpenIndiana VM.
I'm happy enough with the current performance, but I'm always willing to try out something that will provide an increase.
Have you tried DXVA with something else than XBMC? My configuration is quite close to yours, and I am pretty sure that it should also be able to run DXVA.
I was unable to get it working with: Flash, FFDSHOW, W7 default WMF codecs.
Here is my configuration:
- ESXi 4.1
- SuperMicro mainboard (can't remember the ref; I'll check tonight)
- HD 5450 (VMDP)
- Highpoint RocketRaid 3520 (VMDP)
- 24gb of RAM
- Hauppauge Nova-T (VMDP, but still testing)
I shall maybe give a try tonight with a fresh VM. Having a VM as a HTPC would be awesome.
I tried for a bit using HD youtube clips, but I couldn't be sure whether DXVA was actually being used or not.
After doing a bit more research, it seems that DXVA is a very complicated topic, and whether it is used or not depends on the GPU hardware, GPU driver/software versions, decoding software installed, and the individual video file itself.
There is an in-depth discussion here that revolves around a utility called DXVA-checker, available here.
Here's the output from the tool (DXVAChecker64_2.5.0) when run on my system:
AMD Radeon HD 6800 Series
ModeMPEG2_VLD: DXVA2, 720x480 / 1280x720 / 1920x1080
ModeMPEG2_IDCT: DXVA2, 720x480 / 1280x720 / 1920x1080
ModeMPEG2_A: DXVA1, 720x480 / 1280x720 / 1920x1080
ModeMPEG2_C: DXVA1, 720x480 / 1280x720 / 1920x1080
ModeH264_VLD_NoFGT: DXVA2, 720x480 / 1280x720 / 1920x1080
ModeH264_VLD_NoFGT_Flash: DXVA2, 720x480 / 1280x720 / 1920x1080
6719B6FB-5CAD-4ACB-B00A-F3BFDEC38727: DXVA2, -
ModeH264_VLD_Multiview_Avivo: DXVA2, 720x480 / 1280x720 / 1920x1080
ModeVC1_VLD: DXVA2, 720x480 / 1280x720 / 1920x1080
CA15D19A-2B48-43D6-979E-7A6E9C802FF8: DXVA2, -
ModeMPEG4pt2_VLD_AdvSimple_Avivo: DXVA2, 720x480 / 1280x720 / 1920x1080
5B23D46D-FA5F-4FDC-B78A-7EB2787942EC: DXVA2, 720x480 / 1280x720 / 1920x1080
If you click on the big "Check for DirectShow / MediaFoundation Decoders..." button along the bottom of the dialog, you can browse for a test clip. The program should then analyze the clip and present a list of possible decoders to use. If you don't see anything come up in red, then the clip will likely not play using DXVA acceleration. If you do se a decoder that is listed in red, click on the arrow to the left and choose either "play" or "benchmark".
I'd be happy to try some other applications on my system, but I might need some help getting the specific decoder / application chain set up on my system.
Message was edited by: somedude1234 (added missing html link)
Hi somedude1234,
your postings on this topic are greatly appreciated.
Can you confirm that the Radeon HD 3450 also had fully working hardware acceleration when passed through, as you only mentioned gaming, dvxa etc.. with the 6850 card? Im just looking for a card that would work with basic 2d/3d directdraw/direct3d hw acceleration and the 3450 would be a nice and cheap way of achieving this.
Would you also be able to advise of which ATI driver version you used, in case that turns out to be important, or have you used multiple driver versions successfully?
Many thanks in advance.
Regards,
BAM
Hello all,
I have a Dell Optiplex 780 which supports VT-D, and I'm trying to get my Sapphire Radeon HD5450 to work with vmdirectpath.
As far as I can tell, there are no issues passing the device through to the (Windows) VM guest. With no drivers installed the guest boots fine and can see the device, the ATI setup program and windows itself recognise the PNP IDs, and match the 5400 series driver accordingly.
However, once the drivers are installed, I get a BSOD with an ATI DLL "driver in infinite loop" on restarting.
No errors within ESXi/VI client. I have the memory reservation matching the allocated memory, and it's <2GB. The ESXi console video signal drops on VM boot as expected.
What is the best strategy for me here - should I shop around for other video cards to try? is there any merit in trying a radeon 4550?
Or are there any other ways I can work around the ATI driver issue with some other drivers perhaps?
thanks for any ideas
The closest I got to it working was with an nVidia GeForce 7200GS but that also gave the "Driver stuck in an infinite loop" BSOD with nv4_disp.dll. Tried it both on an xp 32bit and win7 x64 vm but same regardless.
The driver used had an impact. I tried various ForceWare versions, but the only that was closest to working was the 181.22 version - this installed, card was listed in device manager with "this device is working properly" status, and I could see it in the display control panel, but as soon as I enabled the monitor connected to it to extend my desktop onto it, the VM hangs. I also tried disabling the vmware svga virtual adapter and this made no difference.
I also tried an ATI FireGL V3100 and it gave various BSODs too. I have just bought an ATI Radeon HD3450 on eBay and am also borrowing a friends ATI Radeon 2400HD so im really hoping I find some success with one of these. I will report back and let you know.
BAM.
OK, just spent the last couple of days experimenting with my friends ATI Radeon HD 2400 Pro and Im happy to say it works very, very well. I have been able to get vga passthrough working with this with zero issues, no BSODs, no crashes, no hanging, pleasantly solid and reliable. Direct3d acceleration, DirectDraw, movie playback, youtube hd via flash (with and without hardware acceleration enabled) all works first time.
On another note, I thought I would share my reasoning behind looking for vga passthrough, as it will certainly not be the same as anyone elses, I suspect
My experiment was to see if I could virtualize a windows pc for the sole purpose of running retro oldskool emulators (such as zsnes, kega fusion, MAME etc..) at native sub-vga resolutions out to an old CRT television via a VGA-to-RGB scart cable. To achieve this on a normal physical pc is complex enough, as it requires very accurate display timing software configuration via the use of an application called PowerStrip. To have all of my existing PowerStrip interlaced and sub-vga progressive resolutions working within the vm via vga passthrough, im seriously surprised and amazed it works, given how sensitive the video output is to very small frequency variations.
I have just received my Radeon HD 3450 today (£12 via eBay!) so I will test that next and let you know what my findings were on that, but for the meantime I think the HD 2400 Pro is a no-brainer for anyone wishing to experiment with vga passthrough as its such a cheap card.
For reference, my config is as follows:
Host
ESXi 4.1u1 + all latest available patches
Asus KCMA-D8 Motherboard
2x AMD Opteron 4180 cpus
32GB DDR3 RAM
VM
Windows 7 x64 (fully patched)
2 vCPUs
2GB RAM
VMDP Devices:
1. ATI Radeon HD 2400 Pro PCIe 16x (works great)
2. Creative Audigy 2 ZS PCI (analog audio output is shrill / crackly, but digital SPDIF out is fine)
3. NetMos / MosChip RS232 (1x DB25 2x DB9) PCI Card (works great)
4. VIA 4-Port USB 2.0 PCI Card
Video Driver Catalyst 11.6 video drivers (as of this posting, the latest available).
Regards,
BAM.
Update: 21/07/2011
Just an update to say ive tested the ATI Radeon HD 3450 and it works every bit as well as the 2400 Pro card. Looks like ATI is the choice to make when it comes to vga passthrough. I also added a VIA 4-port USB PCI card - this is detected as three individual PCI devices, 2x usb 1.1 and 1x usb 2.0. Unless all three are passed through, the vm fails to boot. Only the usb 1.1 ports work in the vm (tested ok with a usb mouse and usb keyboard attached), the driver fails to start for the usb 2.0 controller with an Error Code 10 in Device Manager.
Hi BAM279 Glad to see it worked for you,
I too have some sleepless nights trying to get this to work.... I got from ebay a matrox g200 PCI as I saw the somedude had that built in in his motherboard.
Since I installed the Matrox PCI for the Host VGA, I don't get bluescreens and all my dirvers after install work: nVidia 6200 and 8400GS and ATI HD 4670
(work... they show ok on the device manager)
But I never got it fully working, best I got on one reboot, was to have on Linux the physical screen showing a console (text) and working 100% (text) but not graphics.... never managed to do the same ever again... it works best on XP 32 with my ATI 4670 I see the two external screens of the card on display but as soon as I enable them I the VM freezes!!
With nVIdia cards I sometimes manage to enable the external screen, I can see the mouse going over to it but if I move a window to it it will freeze...
It's so close......
Can you please tell me what is the exact brand and model of your ATI 3450?
Hi,
my specific brand 3450 was one of these: http://www.sapphiretech.com/presentation/product/?cid=1&gid=3&sgid=307&pid=164&psn=&lid=1&leg=1 .
This card also has onboard HDMI audio which is passed through the DVI port and this is indivudlaly listed as a passthrough device in ESXi. I have not tried it as I dont have an HDMI tv, and im passing through my Audigy 2ZS for audio already anyway, but it might be of interest to you if you do... I have also just passed the ATI card and not passed this HDMI audio device and thankfully the card still works perfectly on its own, which means you wont be forced into having to pass this extra device through in case you are tight on free passthrough device assignments (max of six).
Similarly, the Audigy2ZS has three passthrough devices separately listed, the audio card, the gameport and the firewire port - and thankfully once again you can pass just the audio card device through on its own without the others and it still works perfectly (altough as previously mentioned, analog output is horribly distorted, you must use digital out, which is perfect)..
I have been continually using my new vm with vga passthrough without issue over the past few days so it looks like its fully stable I just wish I had been able to get my nVidia 7200GS to work, as it is a bit better for custom resolution output than the ATI, especially with sub vga modes. Maybe with a bit of luck, someone will find a cheap nVidia card that does work
Regards,
BAM.
Sadly, there are still some drawbacks with those old cards.
- You cannot get True HD audio passthrough
- DXVA is older and not as complete as in the new one
From what I understood from this thread, AMD series 2,3 and 6 seems to work with DXVA where the serie 5 don't.
I will manage to buy a HD6450 to see if my guess is correct, if it works it may be an excellent and inexpensive card.
According to the patchnote, some good is also to be expected from the upcoming version 5 of vsphere.
Hi,
I'd like to thank everyone who experiments with vga/gpu passthrough and posts the according system specs! this encouraged me to try it on my system too. here are my specs and results so far:
Motherboard: Asus P8B WS
- Chipset: Intel Cougar Point C206
CPU: Intel i7 2600 w/ iGPU Intel HD 2000
RAM: 8GB (2x 4096MB) Corsair XMS3 DDR3-1333 DIMM CL9-9-9-24 Dual Kit (non-ECC)
SSD: Corsair Force 3 120GB
(off topic: I'm still looking for a HD Tune bench with the same SDD model and the C206 as it doesn't perform as well as it should. Any hint is greatly appreciated!)
ESXi version: 4.1.0.update1-348481.x86_64
VM:
OS: Windows 7 Ultimate 64bit
RAM: 1024MB
vCPUs: 1
GPUs tested:
iGPU Intel HD 2000
worked neither as primary nor secondary (device could not be started)
Zotac Nvidia GeForce 9500
worked neither as primary nor secondary (device could not be started)
ATI Radeon HD 4850
worked neither as primary nor secondary (BSOD atikmdag.sys)
I tried different driver versions and always used a fresh install. I'm also curious and just ordered a 6450 (Sapphire Radeon HD 6450 512MB PCI-E (11190-00-20G) w/ DisplayPort). I'll post whether it works or not as soon as I get my hands on it ^^
cheers
Jason
Hi Jsnow,
Regarding this:
ATI Radeon HD 4850
worked neither as primary nor secondary (BSOD atikmdag.sys)
I had this happen once upon initial passthrough of my ATI HD 3450. Try the following:
1. if your ATI card also has the HDMI audio, pass it through as well, just until you get the catalyst driver installed - it can then be removed again
2. Remove any other passed through devices to the vm temporarily
3. This BSOD is a known issue with the 11.6 driver in general (from what ive read online, so its not necessarly being caused by passthrough)
4. Boot into safe mode, disable the ATI card in device manager, reboot normally, re-enable the card, then try installing the catalyst driver again
Good luck!
BAM.
Hi,
nice idea jsnow.
My Hardware:
Motherboard: Intel DQ67OW
Chipset: Intel® Q67 Express Chipset (Intel Cougar Point)
CPU: Intel i5 2500T w/ iGPU Intel HD 3000
RAM: 6GB (2x 2048MB + 2x 1024MB)
ESXi version: 4.1.0.update1-348481
VGA: Intel HD 3000 + ATI RadeonHD 5450 512MB Silent Edition PCIe (1x Sub-D, 1x HDMI, 1x DVI) with HDMI Audio
VM Details:
OS: Windows XP 32-Bit
vCPU: 2
vRAM: 1800MB RAM (memsize = "1800", sched.mem.minsize = "1800")
VMDP1: ATI Technologies Inc Cedar Pro [Radeon HD 5450]
VMDP2: ATI Technologies Inc Manhattan HDMI Audio [Mobility Radeon HD 5000 Series]
ATI-Driver: 11.5
VMwaretools: VMware-tools-windows-8.3.7-381511
Well...
Windows installed and fully pached. Lastest VMware-Tools installed. ATI-Software installed (without problems).
After restart the Windows-Display-Settings shows 3 Display's (vmware display + 2 from Radeon). If I try to activate one of the Radeon displays the VM stops working vor 1Min and shows an BSOD (ati2dvag.sys 0x000000ea).
I've tried different VMware-Tools versions and a range of ATI-Drivers from 9.1 till 11.6 but always the same damn BSOD.
A suggestion anybody?
exion
Hi,
thanks BAM but unfortunately nothing worked. here's a little protocol of what I did:
fresh install - nothing passed through
installed vmware tools - rebooted, all good -> shut down (created snapshot)
configured both, gpu and hd-audio devices for passthrough:
01:00.0 | ATI Technologies Inc RV770 [Radeon HD 4850]
01:00.1 | ATI Technologies Inc HD48x0 audio
1st try:
windows boot logo, login screen - 2s later BSOD atikmdag.sys (no driver installed yet, Windows used the WDDM 1.1 driver dated 24.04.2009, v8.56.1.15)
booted into safe mode, disabled ATI 4850 card, booted to normal mode
started catalyst setup, during installation: BSOD atikmpag.sys
2nd try:
restored snapshot
booted into safe mode, started catalyst setup => Failed to load detection driver => aborted
3rd try:
restored snapshot
booted into safe mode, disabled VMware SVGA 3D
booted into normal mode, ATI 4850 shows as "This device is working properly" with WDDM 1.1 driver
started catalyst setup, finished successfully => reboot to normal mode
received popup "No AMD graphics driver is installed, or the AMD driver is not functioning properly. Please install the AMD driver appropriate for your AMD hardware."
activated VMware SVGA 3D => computer needs to be rebooted... done
boot into normal mode, windows logo and BSOD atikmpag.sys
4th try:
restored pre-vmware tools snapshot
booted into normal mode, ATI 4850 shows as "This device is working properly" with WDDM 1.1 driver, VMware SVGA 3D is not yet installed
started catalyst setup, finished successfully => reboot to normal mode
received popup "No AMD graphics driver is installed, or the AMD driver is not functioning properly. Please install the AMD driver appropriate for your AMD hardware."
installed VMware Tools => reboot
boot into normal mode, windows logo and BSOD atikmpag.sys
for now I give up and wait for the 6450 to arrive...
something weird though, after each start or reboot, the vga cooler goes full speed 4-6 times in ~1s intervals, on shutdown just 2 times
1s on, 1s off, 1s on, 1s off ... and so on - nothing serious, just thought I'd mention it
cheers
Jason
edit: ok, added one last try and a remark
I do want to know does anybody can VMDp the Intel integrated GPU as primary or secondary output?
I feel interest in it.
Hi,
at least not on my machine
jsnow wrote:
GPUs tested:
iGPU Intel HD 2000
worked neither as primary nor secondary (device could not be started)
maybe someone can try an HD 3000 but I have a feeling that it won't make a difference. Still waiting for my 6450 though...
cheers
Jason
GPUs tested:
iGPU Intel HD 3000 @ Win7-64 and WinXP-32
Waiting for yout 6450 report jason i have ordered one too
exion