VMDirectPath and ATI Radeon

[taylorjonl]

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

[dc_g]

How did you applied U2? Installed on fresh system?

[spangers]

Yeh installed U2 from the Disk.

I think I can pass it through using this method:

http://communities.vmware.com/thread/429305

Any downfalls to doing it like that?

[dc_g]

You could try it.

One question I have you said you can't passthrough devices anymore. Meaning you've done it before with which version worked of ESXi?

[spangers]

The method I posted above worked! Xbox controller is connected.

I got it to work previously with a different motherboard on update 2 but only with the usb 3 controller. This board has no USB 3 controller even listed....

[dc_g]

:smileygrin: good news

Enjoy now

[jhusen]

Hello all,

Thought I'd post my successful build here:

MB:      ASRock X79 Extreme11 BIOS 2.20

CPU:    Xeon E5-1620

RAM:   Kingston valuram 4x8GB 1600 ECC

GPU1:  Radeon HD 5450

GPU2:  AMD FirePro V4900

OCZ Vertex4 used to test swap to host cache, WD Black 1T drive for VM datastore

Highpoint RU1144A card to pass single usb3 ports to VMs

This is running on ESXi 5.0.0 build 914586. I had to inject the nic driver into the ESXi image with ESXi-customizer. I've been really happy with it. Everything passes through (mostly) without issue to the windows 7/8/2012 VMs that I've built so far; I can give each vm over 2GB of ram with the pcihole trick. Youtube green screens on the 5450. Haven't tried linux yet, but plan to once I have some free time. The PLX bridge chips don't get in the way of GPU passthrough on this board.

I also have the LSI controller (reflashed with IT firmware) passing through to a FreeNAS vm without issue with 4x WD Red, 1x OCZ Vertex4 for cache, 1x PNY Prevail Elite for ZIL. I had to use E1000 nics because vmxnet3 is unsupported and I had weird issues using vmxnet2. Seat-of-the-pants performance is pretty good. Haven't had time to benchmark it yet.

The only thing I need to do now is figure out how to get Bluray playback to work for my htpc vm. PowerDVD doesn't want to run in a virtual environment.

-Jon

[derickso]

Hi Jon-

Please update the spreadsheet with your info if you haven't yet: https://docs.google.com/spreadsheet/ccc?key=0Aqp_xYBwP_Y7dE5EclhtaDdIV09lNWxfODd1alRUTlE#gid=0

Also let me know if you get blu-ray playback working, it is on my list as well. I suspect that if you pass through a SATA card to Windows, and hook up the Blu-ray player to that card, it might work. The only thing I could see getting in the way is the VMware SVGA card being the primary card, and PowerDVD being too bad to look for secondary card support of HDCP.

-David

[spangers]

Just checked video playback and it is terrible! I get about 5 fps and super distorted sound that sound slike its playing at .5 speed!

Did anyone have this problem when they passthrough a card and if so how did you fix it?

BTW XBMC menu and programs generally load really quickly and run really fast.

CPU hits 100% during playback aswell....sigh

I also get this messgae in my event log of the VM:

Message from Server: Hardware GPU resources

are not available. The virtual machine will use software rendering.

info

27/01/2013 12:07:35 AM

However Windows RDP (mstsc) plays games/video perfectly from the machine .....

[GBowman]

You have to be really carefull which codecs to use and dont use DXVA its broken.  I use Shark007 codecs and select LAV with DXVA off.  Obviously with DXVA off you need more CPU and 4 are needed minimum for 1080p I have found.  Bluray is something I tried and could not get to work.  WinDVD and PowerDVD will no work in a VM.  I used AnyDVDHD to overcome HDCP but VLC could not play back blurays at a decent rate.  Also DVD playback under a VM fails for me.  I always get a copy protection error and even with AnyDVDHD video is jerky but VLC will play DVD's ok so I use this and dont watch blu rays on my VM HTPC.  Mediaportal is going to support non copy protected blurays in its next release but you will need AntDVDHD or DVDFab running in the back ground to play copy protected material.

You must turn off hardware acceleration in flash to get rid of the green screen.  In windows 7 you can right click on the flash box to get up the settings but in windows 8 you must go into internet explorer options and turn off hardware acceleration for all explorer functions.  I cant use HDMI sound as its just to glitchy and to get the on board sound passed through and sounding acceptable I had to reserve some CPU, I have 2000mhz reserved and this cleared the sound up.  I have a device to mix SPDIF and DVI into HDMI so I will use this when I get time but they are expensive new, its a shame modern GPU's dont have the SPDIF input like the old Nvidia ones have.

[mindless728]

Using a guide to get blu-rays to work with VLC (might not be legal, google search: "Play Blu-Ray with VLC in Windows", second link) I was able to get good playback (ie no missing frames)

I will double check this tonight when I get home from work, but I know it was working without any noticeable skipping.  When I tried to get AnyDVDHD to overcome the protection I got the same problem.

As for the sound skipping, I get the same thing over USB headphones (the entire controller is being passed through). I will try the cpu reservation and see if that helps at all.

[GBowman]

another thing regarding GPU errors in the event log. I get these aswell but they relate to the vSGA addapter, remember the VM does not know you ave a second GPU so it still uses its own video adapter, all my VM's get an error telling me I have too little video ram etc but I ignore it as it will not effect the machines opperation.

If however you want to remove the error about 3D capabilities you must install an Nvidia card and then not pass it through.  I beleive the card must be installed during the install of ESXi for it to be picked up and used for vSGA.

[champagnecharly]

Out of curiosity, would adding this nvidia (why not ati/amd), mean that the processing power of the card is used instead of whatever resources the svga adapter uses? I.e better rendering when svga is in use?

[derickso]

My DXVA works fine with my 7750, at least for decoding MPEG2 used for recorded tv shows in MCE. For Flash/Netflix it works great 99% of the time, but occasionally hard locks the VM requiring a forced reset. Pretty annoying, I've tried the latest drivers and they didn't help.

[Nathanw20111014]

Since you mention using vSGA; are you using this feature (sharing a Nvidia GPU)? I thought this feature wasn't released yet since you need a few things VMware doesn't provide ( namely the nVidia VIB file).

[GBowman]

From what I can tell I have vSGA working in a test server with ESXI 5.0.  I get full Aero in Windows 7 and a WIE score of 3.0 for graphics (2.7 in Windows 8 where the vSGA driver is not used, a generic microsoft driver is used instead).  I can run a few 3D (DirectX not OpenGL) apps from the VMware console but the biggest advantage is that Windows 8 uses the GPU to help remote desktop so with Windows 8 and vSGA there is almost no need for VMWare View or Microsoft RemoteFX.

I beleive only NVidia GPU's work, it certainly did not work for me when I tried an ATI GPU (HD5770) but you must have them installed durung the ESXI install, I had a 460GTX installed and have now downgraded to a 450GTS for power reasons.  I have read about the VIB and as far as I am aware its installed automatically at install if the hardware is there.  I can confirm VM's with a vSGA addapter and DXDiag reporting acceleration although I know there is not a lot of info around on the net so I think its implementation is still not up to a full release standard.

All that said a windows 8 VM with a passed through ATI card is better still, full GPU power to the remote desktop acceleration.  I can run Unigine Heaven benchmark over RDP at 30+ FPS.

Regarding passthru DXVA I can only get hardware acceleration to work when using ATI's own codecs and I cant confirm that is DXVA, with FFDShow, DivX and LAV specifically using DXVA it fails with green screens or no video.  I have only tried a 5xxx and 6xxx cards though.  Its not always possible to select ATI's codecs and even ATI screwed up when they released a new driver version which disabled them so I am stuck with version 11.4 for the moment to keep using good codecs.  Annoyingly DXVA does work with vSGA!

[Nathanw20111014]

Unfortunately it doesn't appear you are utilizing vSGA since ESXi 5.1 is the only version right now that supports it. You also must have access to the nVidia VIB file (it is a separate driver file that must be loaded and is not already installed in the ESXi base image). Refer to this blog post for additional info about vSGA. Right now it appears Quadro cards are the only ones supported.

http://www.virtuallygeeky.com/2012/10/virtualized-3d-gaming-on-vmware-view.html?m=1

[spangers]

I used DXVA checker and it says it is all compatable, I dont have 5k to spend on one of those nVidia cards lol.

Lots of people have got it working, did anyone have stuttering in video playback and actually get it to work?

I keep messing around with it today and get back to you guys.

Thanks

[GBowman]

Yes your right.  I have removed my GPU and the results are the same.  I still have the option to enable 3D support and I still get the same WEI results.  All the applications I tested still work, DXDiag reports hardware acceleration and DirectX11 support with no GPU in the system at all.

I had read so many articles that I was convinced the enable 3D checkbox would only be present if the hardware supported it.  I also read that DXDiag would only report hardware acceleration if it was the case but it is not so (tested in Windows 7 and 8).  Software 3D Emulation is really very good with ESXi 5.0, hardware version 8.

[spangers]

Ok so I figured out the lag was caused by choosing the 'wrong' USB controller type.

I changed the USB controller to xHCI and install Windows 8, bingo Xbox controller works and no video lag.

[The_Dave]

Just wanted to throw my $.02 in. I was able following this thread to get my AMD 5870 passed through to a Win 7 guest using a P55 board and an i7 860! The only issue I see is when I put the machine to sleep it doesn't appear to re-init the VGA card upon waking up so I need to shutdown the machine. Other than that, it is stellar. I installed Borderlands 2 and gave it a go and it runs smooth as can be with maxed settings.

Now I just need to try and up the ram from 2GB to 4 or 8 for Windows. I set it low as I read there was an issue with ESXi in here but didn't recall the fix (PCI hole?) for yesterdays install. I left the VM console vga card in the guest and set to only show the desktop on the 2nd monitor.

Here are the system specs:

ESXi 5.0 u1 (will try with 5.1 here in a bit)

Intel i7 860

Gigabyte GA-P55A-UD4P (I had to contact Gigabyte to request a special BIOS file which enables VT-d)

16GB of ram

XFX OC'd AMD 5870 passed through

USB hub passed through

Nvidia GT220 for the host console

Here was my process:

Flashed new BIOS image

Enabled VT-d by going into new hidden menu (CTRL+F1 <- not mentioned by Gigabyte)

Install ESXi making sure VT-d shows as enabled

Install Win7 as a guest

Install ALL updates, DirectX, .net, java, etc

Set the card and USB to be passed through to the guest and restarted ESXi

Installed the ATI drivers (everything except the CCC)

Rebooted the VM and voila!!

Question: Is there a way to set ESXi not to require a vga card for output? This way I could pass a vid card and my HBA to guest. I am using ZFS as my storage backend and would like either ZFSGuru or OpenIndiana to house my ZFS pools and export out for VM and other storage as well as running a gaming VM.