VMware Cloud Community
taylorjonl
Enthusiast
Enthusiast

VMDirectPath and ATI Radeon

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

814 Replies
jsnow201110141
Contributor
Contributor

I'm sorry to tell, no luck so far on my system... the 6450 just arrived, first it looked promising, no BSOD after passthrough but windows detected the card as "Unknown VGA" and it had an exclamation mark to it ((Device could not be started (Code 10))". During and after installation of the latest catalyst driver I ran into the same old BSOD loop: atimpag.sys *damn*

will give it a more thorough test after I'm back from work.

so long for now

jason

Reply
0 Kudos
FuNK3Y
Contributor
Contributor

Whoups

This is a bad news, I was really hopping that it would work.

I should recieve mine tomorrow, I will do some tests on my own.

Reply
0 Kudos
BAM279
Contributor
Contributor

jsnow,

sorry to hear of your issues with this  Would you possibly have any other passthrough devices being passed through to the host?  Or, do you have another free PCIe slot that you could try plugging the vga card into on your motherboard?

The reason I ask, is because I think that the combination of connected PCI/PCIe devices, the slots they are in, and what ones are enabled for passthrough, can have an effect on whether things work or not.

I came to this conclusion yesterday, after I had added a new PCIe 1x USB controller card and plugged it into my last remaining free slot on my motherboard and set it up for passthrough, only to find that it had "renumbered" my ATI cards passthrough ID from 0:0:7 to 0:0:9 in ESXi, which in turn listed the card as "Unavailable Passthrough Device" in my VMs hardware list.  I removed and readded the card (as the 0:0:9 device) and powered it on, thinking that it should still work wtihin the VM - needless to say, it didnt, and I got the atipmag.dll BSOD error...  I tried reinstalling the drivers and doing the various things I have previously suggested on this thread but to no avail.

I ended up having to remove this PCIe USB card as it wouldnt even work when passed through on its own to another standalone vm - upon powering up the vm I would get various popup errors ranging from "action cancelled by user" to "this vm does not have enough video ram for the specified maximum resolution" which seems bizarre as the vm works perfect as soon as I remove the passed through usb card.

I think this has taught me that ESXi may simply be sensitive to the various hardware configurations we may have and any changes that are made.  I could probably learn more if I started swapping the cards around between slots etc.. but I will leave that for others to test as im just happy my vga card is working again once I removed that usb card!

BAM.

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

Hi,

BAM279 wrote:

Would you possibly have any other passthrough devices being passed through to the host?  Or, do you have another free PCIe slot that you could try plugging the vga card into on your motherboard?

I removed all passthrough devices (e.g. onboard audio) before testing the above. in the VMDP overview, only the gpu and the hdaudio part of the card were passed through. all PCI/PCIe slots on my board are unoccupied but the one with the vga card. I plan on testing the vga card in the other slots as soon as I have time and will let you know how it works out.

cheers

Jason

Reply
0 Kudos
FuNK3Y
Contributor
Contributor

I gave a try with my brand new HD6450, and like everyone I get a BSOD.

I will give another try when ESXi 5 will be released.

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

sorry to hear that FuNK3Y.

haven't had any luck either. I tried several options, different BIOS settings, all the PCIe slots but always BSOD atikmpag.sys

cheers

Jason

Reply
0 Kudos
sevet
Contributor
Contributor

Ordered a sapphire 3450 from eBay, in the meanwhile, I found 3470 at work.

3470 didn't work

3450 arrived from ebay and didn't work as well....  Its almost the same card as suggested by BAM279 the difference is he seems to have a 256mb card and mine which is from the same vendor, has 512mb.... (couldn't find a 256mb) could this be a issue?

This looks so close to working but i'm almost at the edge of giving up, already bought two cards.

What BAM279 wrote in his last post was interesting though regarding other cards in the system interfering with the GPU passthrough, problem is I almost don't have any, I checked with and without a Matrox PCI as the host GPU and passing a second card PCIe (nvidia 6200, 8400GS, ATI 4670, 3470, 3450)

Also with out the PCI card....

I also have a small PCIe SATA controller which I must have tried changing its place, but not removing it.

Any one got an idea?

Reply
0 Kudos
BAM279
Contributor
Contributor

Hi sevet,

Have you set your vm to use no more than 2GB vRAM and fully reserved this amount?

Are you getting the Ati BSOD that many of us have experienced, or something else?  It might be worth trying the trick of disabling the ATI card within safe mode then re-enabling after a normal reboot then installing the catalyst drivers.

BAM.

Reply
0 Kudos
speeedy
Contributor
Contributor

Hi,

I get bsod error about dxgmms1.sys after installing vmware tools. Before tools install radeon 5450 shows correctly in the deivce manager.

What could cause that error. Im using DQ67SW motherboard and Radeon HD 5450

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

Hi speeedy,

I don't know why or how, but I have a feeling that it's related to the virtual vga adapter (VMware SVGA 3D). If you disable this device, the effect is the same as if you hadn't installed VMware Tools. All the passed through vga cards I tested appeared as "working properly" in the device manager. but the respective driver could not detect the device (e.g. ATI Catalyst dialog pops up after reboot, telling "No AMD graphics driver is installed, or the AMD driver is not  functioning properly. Please install the AMD driver appropriate for your  AMD hardware."). after enabling the VMware SVGA 3D adapter, for me, it was always the same = BSOD.

for now I'm also waiting for ESXi 5...

cheers

Jason

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

alright, cards and boards (especially chipsets) on the table - let's visualize what we got here so far:

esxi_vmdp_comparison_1.gif

esxi_vmdp_comparison_2.gif

esxi_vmdp_comparison_3.gif

esxi_vmdp_comparison_4.gif      esxi_vmdp_comparison_5.gif

I kindly ask taylorjonl, NetMika, twood, Zakcar1, Haxxfilif, adiloret and FuNK3: Please post your board/chipset to allow for further comparison.

cheers

Jason

edit 1: hmpf, no cell colors in html tables... damn

edit 2: uploaded screenshots; is it me or are they not being displayed?!

edit 3: hopefully now ...

edit 4: why is there no preview.... geeeeez

edit 5(thousand): finally

edit: thanks BAM, added NV 6800GS PCIe

Reply
0 Kudos
adiloret
Contributor
Contributor

Motherboard: Asus SABERTOOTH X58

Chipset: Intel X58/ICH10R

Graphics cards tested: Gigabyte HD4350, Sapphire Vapor-X Radeon HD 5750

Both cards work fine with basic Win 7 32-bit VM (just installed VMware Tools, attached the PCIe device, and it was detected). Had no issues installing the drivers.

However, as noted by many, DVXA does not work for me.

Reply
0 Kudos
BAM279
Contributor
Contributor

Hey jsnow, nice effort with the compatibility matrix - very useful for keeping up to date on what works and what doesnt Smiley Happy

You can add an NV 6800GS PCIe to my list of tested and -not- working gpus... its another case of error code 10 - device cannot start and has a problem, in device manager.

BAM.

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

thanks adiloret, I updated my post

according to this overview it's salient that you seem to require at least a server-kind-of-class mainboard/chipset to get this to work...

Reply
0 Kudos
taylorjonl
Enthusiast
Enthusiast

My motherboard is a TYAN S7012GM4NR which has the following chipsets: Intel 5520 / ICH10R.  I wouldn't say it worked, it installed and passed through to the VM but it wasn't fully functional.  I havne't worked on it in a while, it is boxed up because I moved, just closed on my new house so I will be pulling it out soon though.

Reply
0 Kudos
jsnow201110141
Contributor
Contributor

thanks taylorjonl!

what do you mean by not fully functional? issues with dxva, 3d accelleration, 2d desktop dual monitor setup or maybe unstable operation?

Reply
0 Kudos
taylorjonl
Enthusiast
Enthusiast

I don't have a lot of details, it recognized, installed the drivers correctly, I was able to display video but when I tried to do something like watch a youtube video, the browser would crash.  I assume the driver didn't do the hardware assist correctly and may be something I could fix but I had to box it up to move.

I didn't have any OS instability.

I may start looking into this again once I move and get settled in but it isn't my highest priority, I instead just built me a $400 HTPC.

Reply
0 Kudos
somedude1234
Contributor
Contributor

Hello again all, apologies for disappearing for a while I've been on a crazy travel schedule recently.

The compatibility matrix is a fantasic idea as there are so many moving parts it's difficult to keep track.

Another piece that I've been thinking about is BIOS settings and all of the different pieces of software involved.  The BIOS settings can have an impact on compatibility, especially when you're talking about advanced features like VT-d.

Also, I think at some point I tried upgrading to Catalyst 11.6 and had to downgrade back to 11.5 immediately due to BSOD.

I'm going to take my rig down to capture the BIOS settings and post back here with the values.

Reply
0 Kudos
exion84
Contributor
Contributor

Hi jsnow,

some adds to the matrix:

- Intel HD 2000 iGPU (@ 2500T, the 3000 was a test on 2500K) -> doesn't work (no BSOD but cant activate second display Smiley Sad )

- Sapphire Radeon HD 6450 512MB -> doesn't work (atikmpag.sys BSOD)

- NV 8800 GTX -> doesn't work (device doesn't start)

- NV 8600 GT -> doesn't work (device doesn't start)

- NV 7300LE -> doesn't work (device doesn't start)

- NV Quadro FX1700 512MB -> doesn't work (device doesn't start)

well... only less ideas left...:

- buy a pcie x1 vga-card to test the other pcie slots

- buy a SuperMicro X9SCA-F (don't know if it works with my i5 2500T, don't know if VGA-VMDP works...) (doesn't support i5)

- buy a  Asus P8B WS (read first, then ask...)

(well... 1 to go^^)

so...  keeps fingers crossed Smiley Wink

cheers

exion

Reply
0 Kudos
danoply
Contributor
Contributor

Hi guys,

As someone who was counting on being able to do this (trying for an HTPC VM) I have to say I was pretty discouraged after reading this thread. Seems like most people have not been successful thusfar.

However, I'm happy to report that I seem to have beaten the odds. My system is a Dell PowerEdge T110, using the ESXi 4.1U1 Dell-customized build. The card is a Radeon HD3450--not sure on exact specs... it was a Dell OEM card I had laying around. OS is Win7 Ultimate x64 SP1, fully patched and had VMWare Tools installed. When I first passed thru the card I got the BSOD via atikmdag.sys (it was using the built-in Win7 driver). So I booted up into Safe Mode and disabled the card in device manager. Then I was able to boot successfully, at which time I downloaded the most recent catalyst driver (I downloaded the driver only--not the full package). Looks like they are up to 11.7 now--released at the end of July.

I installed the drivers--as a part of the install process the installer automatically re-enabled the card. At that point I connected a monitor to the DVI port and boom--there was the extended desktop. I went into display properties and changed the radeon to the primary display and was able to enable aero no problem. It appears DXVA is working because I was able to go into Media Center. Haven't tried a YouTube video yet but I will soon.

I was able to up the RAM to 2816MB (thanks to somedude1234 who posted that value earlier in the thread). I haven't yet tried going beyond that but I will and can post my findings afterwards. But at 2816MB the card works fine.

My hope was to also pass through my Ceton InfiniTV cable card tuner, but two problems arose with that. First of all, the presence of the VMWare display adapter prevents the digital cable advisor from passing due to lack of HDCP. Fortunately there is a workaround to bypass that and enable media center for digital cable anyway. But the bigger problem was that the ceton card doesn't play nicely with VMDirectPath. When you add it to a VM and start the VM, it doesn't even complete the startup procedure--it freezes the whole ESXi box hard--even with nothing else passed through. Sometimes you get a pink screen of death, other times it just freezes on the console display and stops responding period. I've tried it on two different systems (the PowerEdge T110 as well as a Dell Precision T5500) and it does the same thing both places, regardless of what slot it is in, so for whatever reason it just doesn't seem to work properly with VMDirectPath. Plan B for that is to go for the SiliconDust HDHomeRun Prime instead. There is one less tuner but it will be much less problematic because it's a self-contained device with an ethernet interface, so no need to involve passthrough at all.

But I digress, I'm happy to add another report of success using the Radeon HD3450. I have no idea why it works for a couple of us but not others. I really hope vSphere 5.0 has some improvements in this regard, as I would love to give this VM 4+ Gigs of RAM. If I am stuck <3GB then I may actually install the 32bit version of Win7 instead to save a little RAM. We'll see--hopefully 5.0 will be out soon so we can test it.

Next up I need to test the HDMI audio passthru. Right now I just have the card connected via DVI so the HDMI audio device isn't enabled. But soon I'll get a chance to connect an HDMI monitor and that should enable the audio as well.

I have no doubt that passthrough of all sorts of hardware will be commonplace in 5 years. But right now it appears we are on the cutting edge of what is possible in a virtual environment. I hope those of you who haven't had success will keep trying. I will get the hardware details of this HD3450 card in case it helps. And I would encourage everyone to try catalyst 11.7 if you haven't already.

Edit: Corrected my post--of course I meant HD3450, not HD3550. oops. Also tested the HDMI audio--full of pops and crackles. Haven't figured out if that is related to the pass-through because a web search for ATI HDMI crackle gives a ton of results of that issue even in a non VM environment. Worse comes to worst I think I can use a USB sound card but I prefered the ATI solution since it injects the audio into the HDMI connection. Will have to do some more troubleshooting as I guess it could also be the fault of the TV I tried it on.

YouTube worked fine although I had the major audio crackles via the HDMI audio.

Also I have more specs on the card. It is a Pegatron RV620LE (HD 3450) with 256MB of RAM (VEN_1002&DEV_95c5). It is low-profile and has a DVI and HDMI connector. Unfortunately, I believe it is not HDCP compliant because it is giving me troubles trying to watch protected content in media center. But again that could be an issue with the cable and/or display device.

The driver I'm using is 8.872.0.0, dated 7/7/2011.

I have access to some other Radeon cards so I will plan to try those in the coming week. Will keep you all posted as to my results.

Reply
0 Kudos