VMDirectPath and ATI Radeon

[taylorjonl]

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

[akuznetsov]

Hello !

Just to report success and failures.

System:

Supermicro X9SRL-F

Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz

32GB ECC RAM

a lot of SATA HDDs

ESXi 5.5.0

Hardware for pass-through:

Turks PRO [Radeon HD 6570]

uPD720201 USB 3.0 PCI Host Controller

Creative SB SB1040

Results of pass-through:

uPD720201 USB 3.0 PCI Host Controller - passed OK under Win7, Win8, Gentoo Linux, Ubuntu Linux. But it's impossible to use them with web-cams (the main purpose for them was home CCTV). The work is very unstable with one web-cam. The second web-cam isn't accessible at all.

Creative SB - everything is OK under Win7

Radeon HD 6570 - failed to install drivers under Linux. Drivers installed OK under Win7/8, but hypervisor constantly falls into PSOD (it's very very bad for another 17 VMs on it...).

I'm searching for stable Radeon card for now (gonna replace desktop with one VM with Videocard and USB-card passed through).

[heatvent]

afarrugia and/or bundyboyuk,

I think I am having a similar issue to you.  Audio plays but somewhat distorted/echo like there is feedback or the audio is looping or something. My setup is:

ASRock 970 Extreme3 R2.0 - Bios 1.60

ESXi 5.5 u2

Radeon HD 5450 card / 5650 card

HDMI -> receiver -> TV

Allocated memory is locked

Have you had any luck fixing?

I have tried everything under the sun to get this to work...

Downgraded to ESXi 5.1 u3

Tried 3 different HD 5450/5650 cards

Uninstalled re-installed everything

Moving positions of cards/slots

I'm starting to think it's my motherboard although it is the same MB as used in this build (http://thehomeserverblog.com/esxi/esxi-5-0-amd-whitebox-server-for-500-with-passthrough-iommu-build-...).  I am pretty sure at one point I had these same cards passed through albeit perhaps on a MSI motherboard.

Anyone with any further suggestions let me know but I think I have eliminated most if not all possibilities.

Thanks!

[bundy_boy]

Hi Heatvent,

No I haven't been able to fix it so far...it comes up after about 1 hour of playing video or music, and then it is more and more frequent...I am pretty sure I did not have this when I was running 5.0 though

[wings7351]

I made a post then I found this thread, should have searched more thoroughly first...

Here is my situation, I am hoping to find someone with similar experience to help me with my situation.

I am setting up a home lab with DZ77GA-70K and the vt-d capable Xeon CPU. In my setup, I have a LSI2008 card and an AMD 4350 video card, I am trying to passthrough both cards to two different VMs. This motherboard has two x16 slots, the primary one is x16 speed, the secondary one is x8 speed. I find that:

     1) if I put the video card into the primary one, the LSI card into the secondary one, the LSI card does not show up in the passthrough options

     2) if I put the LSI card into the primary one, video card into the secondary one, both will show up and passthrough, however the video card does not function properly and shows up in the windows 7 vm device manager as "has a problem and stopped".

Does anyone have similar experience? I know none of these hardware is officially supported by Vmware, but I am hoping someone can share their experience and knowledge on this.

Thanks.

Edit: forgot to mention, I am on the ESXi 5.5.0.

[derickso]

Get a newer video card, search through the thread for an exact model #

of a card that has been confirmed working, and buy it... they are very

finnicky.

[wings7351]

Tested with XFX 7970:

1) 7970 on primary slot, cannot see the LSI card on the secondary

2) LSI on primary, cannot see 7970 on the secondary

Tried to disable the onboard audio, the system won't even pass the post, very strange.

[derickso]

Make sure your motherboard has both of those slots enabled, can you see

both of the cards if you boot something else like Ubuntu?

[wings7351]

Thanks for getting back to me.

I could see both slots and hardware when I plug LSI into primary slot and ATI 4350 into the secondary slot. The problem is that in this configuration the ATI 4350 won't work properly for unknown reason.

Does this answer your question about both slots enabled?

[derickso]

Yeah, could be a motherboard compatibility problem, what board do you

have? And is it updated to the latest bios?

Also what CPU?

[wings7351]

My CPU is Xeon E3-1230 v2.

I have found part of the issue. I read a thread on Intel forum talking about how the PCI-E lanes are configured. Essentially on this DZ77GA-70K, between the 2 PCI-E x16 slots and one x4 slot, the total lanes can only be 16. It can be x16, or x8+x8 or x8+x4+x4. The last configuration is used when the x4 slot is used. I have a Intel server network card on that slot. That is why the system can only see LSI card when it is put into the primary x16 slot. Tested by removing the x4 network card, proved this to be true.

However the ATI 4350 GPU passthrough is still not working as expected no matter what configuration I put in. I know this card worked when I was using an AMD board for my server.

[wings7351]

OK, based on my last post, took my x4 network card out, if 7970 on the primary slot, still won't see the LSI. Moved the 7970 to the secondary slot, LSI on the primary slot, both seen in the list and got passed through.

So now 7970 is working in x8 mode, based on Intel manual. Assigned the 7970 to a Windows 7 vm, screen shows up no complaints, but my mouse and keyboard input are out of control. I cannot see my mouse cursor, right or left click on the mouse, a popup window (for screen resolution) will show up somewhere else on the screen. Any idea what is happening?

[derickso]

Did you pass through your usb controller as well?

[wings7351]

No I did not. Are you saying the mouse and keyboard input from the ESXi console won't work in this case?

[heatvent]

I think he is saying that but I am guessing your issue is that your mouse is traversing over two extended screens but the ESXi remote desktop doesn't seem to adjust to this well.  I have had the same issue so I use something else to get to the VM rather than ESXi client.  I was using Teamview which works well but leaves popups upon exit which I didn't like for my HTPC VM, so I am using Remote Utilities which is a little less user friendly but works well and has some nice features.  Depending on the target OS you could use RDP or some people use VNC for remoting in. 

Sorry, one final alternative that I also use is I access through the monitor/tv attached to the video card and disable the VMWare virtual monitor in Windows.  This works well but there is some risk if you have issues with the actual passed through card, you may have difficulties getting back into the VM if ESXi client is cut off. 

[derickso]

Yeah my VM connects directly to our TV and I have a dedicated USB

controller passed through to windows that I plug a wireless

keyboard/trackpad into. I also do disable the ESXi Vga card. I haven't

had an issue with reverting back, anytime I unplug my monitor from the

physical video card it turns the other one back on and I can get to it

via the esxi console.

[wings7351]

So I installed VNC without the GPU, enabled the GPU passthrough after that; connected to the VM via VNC, found out that the VM (Win7) automatically chose the GPU as the primary display, which is why I couldn't have input in the ESXi console.

I then use Teradici PCOIP host adapter passthrough with a zero client at the other end to get this working.

Everything seems to be working now. Thanks.

[spoonuk]

Can someone summarise which ATi cards are confirmed working for PCI pass-through and on which versions of ESXi?

I'm having issues with error 43 on GeForce 750Ti but that's another story...

[KRA77]

Someone did, there is a spreadsheet if you browse back some pages.

[mindless728]

Here is the spreadsheet with everyone who has added their data to:

https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE&usp=drive_w...

Some of the data (like mine) might be a bit dated because it has been a while since we added it

[spoonuk]

Perfect, that's what I was looking for!

Time to get some ATi cards in as this GeForce 750Ti is driving me nuts...