VMware Cloud Community
taylorjonl
Enthusiast
Enthusiast

VMDirectPath and ATI Radeon

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

814 Replies
homerjr43
Contributor
Contributor

Looking for advice,

I have a server running multiple VMs.  Freenas, and two instances of Windows 7.  Hardware includes two Xeon 5520's, 36GB of ram, 2 6450 (one in 8x slot and one in 16x slot), and two Intel M1505 sata cards.  Each Win7 VM has USB ports and a ATI 6450 with passthrough.  I use these VMs as KODI/XBMC frontends.  Both machines are working and the passthrough and drivers seems to be properly installed.  However, I am having a very frustrating issue.  Overtime, the video output and/or audio output gets choppy or ceases to work.  Restarting the VM does not resolve the issue, but if I reboot the entire server, the 6450 work again.  This is very frustrating because rebooting the server properly with the three VMs takes at least 30 minutes.  I have reserved the memory for all three VMs, but I have not done the PCI hole end/start config.  Would this resolve this issue?  If not what could be the problem?  After alot of research, most people it either works or it doesn't.  I have not been able to find any suggestions about my issue. 

Thanks

Reply
0 Kudos
bundyboyuk
Contributor
Contributor

Hi Homerjr43

I already posted about a very similar issue in this thread. I too had an issue where after playing a video or music for some time (it can be 2mins or 1h00) it would slow down and get completely distorted. I have an AMD config with a 7750 AMD graphic card. I never managed to fix this so since I was getting really irritated, I am now using a real machine for watching videos and playing music...

Bundyboyuk

Reply
0 Kudos
homerjr43
Contributor
Contributor

Bundyboyuk,


Thanks for the update...Not really what i wanted to here.  The only difference with mine is that its usually over a few days, not minutes or hours.  Just odd, that it works great, then appears to get bogged down.  I was hoping that it some configuration or setting could be changed to make it more stable.


Reply
0 Kudos
babolcs
Contributor
Contributor

Hi David,

Did finally work the new SATA card attached BluRay player solution?

Or have you found anything other.

I'm fighting to have a working BluRay player in a VM too (ESXi 6+Win10) +LG player, no dedicated SATA card passed through to the VM, just the optical disc by ESX host itself.

Thanks,

Szabolcs

Reply
0 Kudos
JohnOCFII
Contributor
Contributor

Glad to have found this thread -- I hadn't looked here, as it's in the ESXi 4.x section.

I used the information to create a Home Server from Don's blog: http://thehomeserverblog.com/esxi/esxi-5-0-amd-whitebox-server-for-500-with-passthrough-iommu-build-...

It all worked fine for 18 months passing through an ATI Radeon HD 6450 to Windows 7.

I recently upgraded from ESXi 5.5 to 6.0, and it continued to work for two weeks.

Last week, after applying Windows Updates, my Windows 7 VM user interface began to lock-up.  The VM would boot OK, but output via the passed-through graphics card and input passed-through USB card (for keyboard and mouse) would freeze.  I had some external network WMI monitors running, and it was obvious the VM was still running.

I tried a few things, including building up a whole new VM with Windows 7.  It fails in exactly the same way.  If I remove the passed through video card and USB bus, it works fine.

I do not currently have any manual configuration items set (pciHole, etc.).  The 3 GB of RAM I have allocated to the VM is reserved in the GUI.

Any suggestions?  Are there some manual configuration tweaks I should try?  I did also create a post in the ESXi 6.0 section but no replies over there, and it seems all the passthrough action is in this thread.

Thanks

John

Reply
0 Kudos
kvitaly2005
Contributor
Contributor

Have same BSOD problem with same HD 6450 on i5-2500 and Intel Q67 with ESXi 6.0 while trying to make it work under windows 10 VM.


I had this card working on ESXi 5.0 and Xeon 1230 with Windows 7 for a while, but after processor upgrade (to make use of vPro KVM) with ESXi upgrade it just gives VIDEO_TDR_FAILURE inside VM every time it try to activate the screen.  Is 6450 some particular model to avoid? Or all Radeons is broken completely in newer ESXi?  Will NVIDIA work?

I tried to disable audio, internal graphic. No other devices attached to slots.

Thanks for sharing!

Reply
0 Kudos
ToddD
Contributor
Contributor

I will give a little information about my setup, I have a Dell PowerEdge T710 with (2) XFX R5 2XX (HD6450) GPU's with (2) Jaton USB3.0 PCIe add-ons. I have (1) GPU and (1) USB add-on passed through to a Windows 10 guest, and the other GPU and USB add-on passed through to a Linux OS. ESXi 6.0 Host.

The Win10 environment works flawlessly. The Linux OS on the other hand I can't seem to get it to light up the heads. I have tried Ubuntu/Fedora/RH/Gentoo/ArchLinux and even a custom linux build. In all of them I can get the kernel to detect the card, compile the catalyst drivers and have them recognize and the card, but I cannot get Xorg to start because it seems to think there are more heads on the card than the card can support.

I am talking to the Vi0L0 (whom is the unofficial catalyst portage builder for ArchLinux) to see if it is something stupid happening between the vmwgfx driver and fglrx driver. Was hoping this thread had some interesting insight, but it appears not so much. If anyone develops some work arounds, let me know.

Reply
0 Kudos
haerrir
Contributor
Contributor

Hi all

I have an HP ProLiant DL380p G8 currently for test purpose.

I've installed ESXi 6.0, the HP Custom Image.

And i put an AMD R7 250X in the server.

Passtrough is configured and it works, GPU is recognized as a Radeon HD 7700 GHz Edition.

I created a Win10 VM with the VmWare Tools, PCIHole min and Max Settings in the vmx and the newest AMD Drivers.

The GPU is recognized inside the VM as an R7 250X. But i can't run any Games, Blackscreen or freeze, even FurMark starts with an Blackscreen and after some time a picture comes but it's frozen. Also GPU Shark shows 0.0% GPU Usage.

Any ideas what could be missing or be wrong?

Also sometimes the hole vSphere Console freezes or the Server restarts it self.

Reply
0 Kudos
esxijoe
Contributor
Contributor

My ESXi setup is having issues with GPU passthrough and linux.

Setup:

AsRock H97M Pro 4

Intel Core i7-4770

16GB RAM

ESXi 6

After much frustration to get GPU passthrough to work, I had to set the primary GPU to PCIe and disable onboard audio as well.

First started with a Nvidia 750Ti card - loads of frustration only to realize that Nvidia cripples(shame on you Nvidia) Geforce drivers to not work inside of VMs. Got a Radeon R7 360. Installed Win7 SP1 64-bit and everything just works great. So installed a second card HD6670 to pass to linux - first tried with Windows and it works great. Next tried with various linux distros and all of them complain about BIOS ROM not being found and bogus alignments. Tried pciHole settings and still no luck. Tried various RAM sizes and still no luck. Tried with HD5450. Same issue. Works great in Windows but complains in linux.

Anyone got a workaround for the BIOS ROM issue?

Reply
0 Kudos
esxijoe
Contributor
Contributor

How to fix choppy/metallic HDMI audio issue:

Didn't notice this until last night and the fix that worked for me is pciPassthru1.msiEnabled = "FALSE"    in this case HDMI audio is assigned as PCI device 1 so make sure you find the right number in your configuration.

I also tried one other tweak but that didn't help in my case but it may work for you. You can find the solution in post 323 in this thread by user derickso.

I also tried passing thru onboard HD audio. Win7 detects it but after that anything to do with audio playback comes to a grinding halt.

EDIT: I tried passing through the onboard audio again by disabling MSI as above and it's been working great so far.

Reply
0 Kudos
GLRoman
Enthusiast
Enthusiast

Any luck with this?  I'm getting the same errors as you:

Mar 25 15:41:05 test9 kernel: radeon 0000:13:00.0: enabling device (0000 -> 0003)

Mar 25 15:41:05 test9 kernel: [drm] initializing kernel modesetting (OLAND 0x1002:0x6610 0x1787:0x2012).

Mar 25 15:41:05 test9 kernel: [drm] register mmio base: 0xD2F00000

Mar 25 15:41:05 test9 kernel: [drm] register mmio size: 262144

Mar 25 15:41:05 test9 kernel: radeon 0000:13:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)

Mar 25 15:41:05 test9 kernel: radeon 0000:13:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x20000000] (bogus alignment)

Mar 25 15:41:05 test9 kernel: [drm:radeon_get_bios [radeon]] *ERROR* Unable to locate a BIOS ROM

Mar 25 15:41:05 test9 kernel: radeon 0000:13:00.0: Fatal error during GPU init

Mar 25 15:41:05 test9 kernel: [drm] radeon: finishing device.

Mar 25 15:41:05 test9 kernel: [TTM] Memory type 2 has not been initialized

Mar 25 15:41:05 test9 kernel: radeon: probe of 0000:13:00.0 failed with error -22

My hardware:

Mobo:       ASRock Z97 Extreme6
CPU:        Core i7-4790S
Video:      HIS H250FS2G Radeon R7 250 2GB
HW Level:   vmx-09
ESXi:       5.5 u3b
Guest:   Fedora 23 w/2GB

(CentOS 7 fails the same way.)  I've disabled onboard graphics and audio.  I've tried re-loading the radeon module in Fedora with runpm=0, pcie_gen2=1, and msi=0 but no joy.

Reply
0 Kudos
MrPippy
Contributor
Contributor

To fix the BIOS loading problem, I patch the kernel Radeon driver to load the VBIOS from a file in case the other methods fail. I ended up just downloading a copy of the VBIOS for my card, but it's safer to dump it out from DOS.

Heres the patch (might need some changes for your kernel version): https://lists.freedesktop.org/archives/dri-devel/2014-November/072701.html

Reply
0 Kudos
GLRoman
Enthusiast
Enthusiast

@MrPippy: Thank you!  Your link shows how to modify the source code of

my kernel, but rather than go that route, it gave me enough info to find

the ATI site where they have posted RPMs for various cards.

I did the following on my CentOS 7 box:

- I downloaded https://www2.ati.com/drivers/linux/fglrx64_p_i_c-15.302-1.x86_64_rhel7_64a.rpm

- I installed all the missing dependencies (note: you also need redhat-lsb-core)

- Then:

  % rpm -Uvh fglrx64_p_i_c-15.302-1.x86_64_rhel7_64a.rpm

  Preparing...                          ################################# [100%]

  Updating / installing...

     1:fglrx64_p_i_c-15.302-1           ################################# [100%]

  /usr/libexec/plymouth/plymouth-populate-initrd: line 233: dfatal: command not found

  /usr/libexec/plymouth/plymouth-populate-initrd: line 234: dfatal: command not found

  /usr/libexec/plymouth/plymouth-populate-initrd: line 235: dfatal: command not found

  /usr/libexec/plymouth/plymouth-populate-initrd: line 236: dfatal: command not found

  /var/tmp/rpm-tmp.ahgIOH: line 1171: lsb_release: command not found

  ....

  <reboot>

  % aticonfig --initial

  % startx

Then I added a wireless keyboard and mouse plugged into a USB card

that's also passed through to the VM.  Success!  I can run `startx` from

the console and X11 runs on the Radeon display.

Now I've advanced to the next problem.  When I change the system default

to graphical:

  % systemctl set-default graphical.target

and reboot, I get a screen that says, "Oh no! Something has gone wrong."

So the problems continue ....

Reply
0 Kudos
ciuly
Contributor
Contributor

It appears someone has modified the sharing access to the google docs spreadsheet on this subject. I have restored full access to everyone and removed all editors plus prevented editors from changing these settings again. Hopefully all will be good now.

VMDirectPath with ATI GPU document https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE
Reply
0 Kudos
edward_walton
Contributor
Contributor

I can't seem to figure out how to post on the thread without directly replying to someone.

Anyhow, I've got a PowerEdge T320 on the latest firmware.

Installed ESXi 5.5 U3.

Created a Win10 VM.

Passed through my AMD FirePro W7000 and USB 3.0 PCIe cards.

Got their drivers installed.

And bam, I get working video/mouse/keyboard and even "audio".

But the AMD HD Audio is only giving me stereo.

I'm not getting any Dolby Digital, DTS, etc.  Nothing but stereo?!?

Any suggestions?

I'll be trying a Win7 VM soon, but those take forever to load up all the latest MS updates before I can really test it.

Reply
0 Kudos