VMware Cloud Community
taylorjonl
Enthusiast
Enthusiast

VMDirectPath and ATI Radeon

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

814 Replies
twood201110141
Contributor
Contributor

Hi TheGrave,

In fact you can click inside the VI console of the Virtual machine, and then switch your monitor to your VMDP VGA, (assuming you have assigned a VMDP vga to your VM) and you will actually see the cursor moving there which is very funny considering you "left' your input to the VI console.

This is very buggy though (slow moving and jumpy cursor ) and you also need a second PC to launch your VI console.

So the best way to do this, as Dave mentioned, is to VMDP a couple of USB ports to your Virtual Machine and attach there a pair of keyboard and mouse.

Now regarding both inputs and sound output In my setup I have my ESXi 4.1 running happily, and i have VMDP one usb port (this is because I have a PC along with my ESXi and i use a KVM which has the keyboard and mouse attached) one ATI 5450 VGA adapter, one creative labs soundblaster audigy and the onboard firewire which I have not tested yet.

Input, sound and graphics all work great. For the sound I couldn't VMDP the onboard sound card as it was crashing my ESXi for some reason, so I just plugged an old good soundblaster audigy that was hanging around Smiley Happy

Regaring which usb port to VMDP which actually means which of the usb controllers to use, you will have to do some trial and error until you find the right ones. I recommened to try one or two usb controllers each time and not all at once.

regards

Theo

Reply
0 Kudos
twood201110141
Contributor
Contributor

This is very interesting, I will test and post my results.

It is very strange indeed that there is nothing in VMware knowledge base regarding this problem with VMDP and configured memory

Reply
0 Kudos
twood201110141
Contributor
Contributor

Well what the guy mentions in the video is that you need to have memory reservation on the vm that has vmdp devices assigned, equal to the vRAM configured, otherwise it will give you an error on power on.

This is a well known prerequisite to run a VM that uses VMDP devices.

Our porblems with the 2GB limtitation begin well after that.

The problem is well described on the first posts of the thread.

I wish a vmware representative come to this thread and give us his lights!

Reply
0 Kudos
TheGrave
Contributor
Contributor

Great news, I'm planning to attach my good old Audigy 2 ZS as well. I don't have an extra laptop at home for my parents so I need the machine to be usable from the console, that's why I'm planning such twisted setups:)

One thing I don't understand though - if you VMDP a device is it directly accessible by all virtual machines at the same time (this would mean shareable DMA, IRQs and other stuff that would decrease the performance significantly) or just one of them has direct access and the others see the VMWare virtual adapter?

Reply
0 Kudos
twood201110141
Contributor
Contributor

Well the way I understand this is that when you VMDP a device you are declaring it as a device that can be used directly from a Virtual Machine and therefore bypass the vmkernel for that device. It is like taking the device from the ESX server and having it ready for passthrough. At this point nothing really happens except of course that ESX server can no longer access the device, so be careful what you choose there as you may ruin your ESX installation. (for instance VMDP the sata controller where your ESX boot disk relies!!!)

You will also need to reboot the ESX server after configuring devices for passthrough.

So after that you will have to assign the VMDP device by editing the target VM's configuration and add the device found under "PCI devices". You can add up to 6 VMDP devices and of course you cannot have more than one VM's sharing the same VMDP device.

This will benefit you with performance but will sacrifice some of the nice features of ESXi hypervisor such as snapshots, hot plug disks or nics etc.

And of course will bound you with the need to reserve all the memory configured on the VM and the dreadfull pci.hole error when you try to give more than 2GB RAM to the VM.

It is a nice feature though this bug seems to spoil the fun.

On the screenshots attached you can see my VMDP configurations

Reply
0 Kudos
mikelane
Expert
Expert

I have read that directpath does not support hardware acceleration with video cards - so I am wondering what exactly the use cases for a directpath video card would be? I'd love to virtualize my media center but I am assuming that streaming 1080p will be a no go with directpath?

Reply
0 Kudos
adiloret
Contributor
Contributor

I was finally able to get this to work with Win 7 32-bit with only 2GB of RAM and a Radeon 5750 1GB.

I am curious as to why video acceleration is not supported. With the latest flash player stuff like youtube videos hang with a green screen. I also see the same thing with DXVA video acceleration with videos. After disabling hardware acceleration for both, everything works fine again (of course producing a higher strain on the CPU). Does it have anything to do with the fact that there are two video adapters on there and the VMware video adapter doesn't deal with the acceleration request properly? Is there any way to completely remove the virtual video adapter so the VMDP adapter becomes the primary?

I also have one other hiccup, but I shouldn't be surprised. I chose to VMDP the primary video card used by ESXi 4.1, so the console video gets ripped away by the VM when it boots up. I can deal without consle video, but when I go to reboot the ESXi server, near the final stages before the reboot it just hangs. When I reset the machine things are back to normal again. I guess I could trace it to see what is actually happening.

This whole idea of VMDP is pretty impressive. I just hope VMware works to further improve/stabilize it so it can be used with many more PCIe devices.

Reply
0 Kudos
sowdesh
Contributor
Contributor

Hi twood,

I too am trying to configure Radeon HD5450 video card as a VMPD device in Windows 7 guest.  I added both the video and audio pci devices and installed the ati device drivers in win7.  Both of these devices are active in device manager.  But the problem is there is no video output to this card.  In Windows 7 video settings, only the vmware vga is displayed as device and the ati video device is not listed.  I'm trying to use the same video card as vmpd for this vm that is also the primary for vmware.  I get the vmware startup on the display and once I start the vm, the display goes blank and says no signal.  The same has been experienced by Haxxfilif as he said in this forum.

My question to you.  What guest OS are you running with successful video output on HD5450.  Do you use this adapter as both primary for vmware and configured exclusive for a guest OS or do you have more than one video cards?  Could you also give me some more information on the hardware its been installed?

I have the HD5450 installed on a Dell PowerEdge T110 server.  It sits on the pci-e slot 2 as its the only slot with x16 length that could accept this card.  Its got 1GB video ram.

Any and all help will be appreciated.

-S

Reply
0 Kudos
twood201110141
Contributor
Contributor

Hi sowdesh,

I'm also using one VGA card (ati 5450) for both ESX server as well as a VMDP device.

My ESX 4.1 server is running on an Asus whitebox mobo (P6X58D-E) and the guest OS with the VGA assigned as VMDP is a windows 7 64 bit machine.

I have also tried with windows XP and windows 7 32 bit and is working there too. I am just limited to 2GB vRam allways.

So anyways back to your problem, I think that what you describe is normal up to a point.

In the beginning when i initially configured my video for passthrough I too noticed that as soon as i powered on the VM which has the passthrough video assigned, the ESX server's console went blank.

Back to my VI console I went to the screen resolution settings and first i had to select "extend these displays" and then "show desktop only on 2" which is the VMDP display as it seems that the vmware adapter is still the primary one.

After that everytime i power on this VM the ESX console is redirecting to the VMDP display and the VI console just shows the win7 boot logo but this is fine as everything is being displayed directly to my monitor and this is so cool actually.

Makes the VM feel like a native machine.

So try to hit detect under screen settings and then choose "extend displays" hit apply and then "show only on 2" and apply again.

If nothing happens there then I can't really know what could be wrong with your setup.

You will also need to VMDP a usb controller too so as to have native imput on your VM.

Reply
0 Kudos
FuNK3Y
Contributor
Contributor

Hello,

I also gave some try to this delightful unsupported feature. I added a second GPU in my server, and tried to dedicate that one to a VM (in order not to mess with the console).

Sadly, with this configuration I was unable to redirect the VM display to the dedicated GPU. It appeared in the Device Manager, but it acted as if no display was attached to it. When I tried to update the driver, it sometime couldn’t start (the driver).

I tried with an nvidia 6200 and the well-known ATI 5450.

Has anyone made some new test since the Update 1 for ESX has been released?

Reply
0 Kudos
somedude1234
Contributor
Contributor

I've been successful with both an AMD Radeon HD 3450 as well as a Radeon HD 6850.

The ESXi host is a SuperMicro X8SIA-F (Intel 3420 platform) with a Xeon X3440 and 16GB RAM.

The VM that's getting the graphics card is Windows 7 Ultimate x64.  Both the older (3450) and newer (6850) cards actually present two PCIe devices.  The first is the GPU and the second is an audio device (for audio over HDMI / DisplayPort).

In addition to the AMD GPU and AMD audio device, I'm passing through one of my two Intel USB host controllers (onboard device coming off of the 3420 PCH).  This lets me connect my keyboard, mouse, webcam and USB sound card for a fully functioning workstation as a VM.

I can confirm that any VM which has VMDirectPath devices must have 100% of it's configured memory reserved (this is the reason why I upgraded the host from 8GB to 16GB of RAM).

I am also running into the RAM limitation.  It took quite a few reboots of the VM, but I've determined that the RAM ceiling for my workstation VM is 2816 MB.  It's an odd number, but if I go any higher then Windows will quit at boot with a random BSOD.  It is very frustrating that I can't configure the workstation VM for a proper 4-6GB of RAM, but it's a tradeoff I'm willing to make for this setup.  Hopefully a future release of ESXi will address this limitation, but I'm not holding my breath as this setup is definitely not supported.

I initially tried using USB passthrough for the keyboard and mouse, but they didn't even show up in the list of available USB devices to pass through in vSphere.  It seems only USB storage and hardware locking keys are supported for USB passthrough.  Giving the VM direct access to the USB controller has been a great workaround.

Reply
0 Kudos
somedude1234
Contributor
Contributor

Some tips for anyone else trying this out:

1. Make sure that you are using the WDDM 1.1 driver for the VMware SVGA display adapter.  There is an older, non-WDDM driver but Windows 7 doesn't support heterogeneous display adapter setups unless all of the adapters are running WDDM 1.1 drivers.

2. I ended up disabling the desktop display on the VMWare adapter.  This will prevent you from accessing the console from vSphere, but I use remote desktop for situations when I'm not at the local console.  The advantage is that your mouse won't disappear onto the extended virtual screen when you're at the local console.

3. Start off at 2GB of RAM and then work your way up if you're stubborn like me and insist on > 2GB.  Make sure you update the memory reservation whenever you increase the amount of configured RAM for the VM.

Reply
0 Kudos
zweigoh
Contributor
Contributor

No hardware acceleration for VM using VMDP PCIe passthrough?

Does this mean the VM can't play Youtube HD movies on it? Is the video output from the VM onto the monitor appear smoothly or jerky?
Hope you guys can enlighten me.

Thanks

Reply
0 Kudos
adiloret
Contributor
Contributor

Unfortunately no hardware acceleration. Smiley Sad

However, I think the answer is it depends on your video card and CPU. With my ATI 5750 I have no problems whatsoever watching 1080p YouTube or any other video for that matter. I'd imagine any mid-level video card and decent CPU will do just fine.

One thing to watch out for though, is acceleration gets turned on by default in new versions of flash if they detect a compatible video card. You need to shut this off before trying to watch a video or it may hang.

Reply
0 Kudos
somedude1234
Contributor
Contributor

Hardware accelerated h.264 playback (DXVA2) is working for me using the system described in an earlier post (AMD 6850).

I used XBMC as my test bed.  There is an easy method to enable/disable accelerated playback in XBMC and you can confirm whether it is enabled by pressing 'o' during playback.

WIth hardware acceleration enabled, my CPU will be at 3-5% load during playback of 1080p video and it will be perfectly smooth.

When I disable hardware acceleration, CPU usage will range from 30 - 50% and on certain videos playback will skip badly.  I'm not sure why this is as the CPU should be more than capable of software-only playback.

I am also able to play games with native GPU performance, including three 1680x1050 displays running in an eyefinity setup.

I'm wondering if the DXVA issues that others were seeing are a simple driver or software bug that is completely unrelated to ESXi and VMDP.  I read that AMD had to update their display drivers after Adobe enabled DXVA support in flash.  For anyone who is trying to make VMDP work with DXVA and having troubles, I recommend ensuring that you try the latest versions of flash as well as your display card drivers (if you haven't tried this already).

Reply
0 Kudos
cipcm
Contributor
Contributor

Hi,

first I want to thank for this brilliant solution. This is the only way I found so far where you can use a VM like a normal PC without any further hardware.

Now what I am conerned about is how I do upgrades and patches. If I have an upgrade requiring the ESXi host to go into maintenance mode this will leave me with no display. As long as I still have a second computer I will be able to do the upgrade from there and put the ESXi into normal operation mode again.

But what to do without a second pc? Maybe one could do the update with a script (ssh to ESXi host, put a script there which shutdowns all vm's, goes into maintenance mode, upgrades, leaves maintenance mode and reboots). If everything goes well you will be back in a few minutes. However as we all know things tend to go wrong. Therefore I'd wish to have a view on the ESXi console when doing the update.

Is there a way to disable VMDP for the graphics adapter from within a running vm? Or maybe can I alter a config file and then reboot without VMDP for graphics?

Regards,

cipcm

Reply
0 Kudos
FuNK3Y
Contributor
Contributor

I was able to add a second GPU to my ESX host, that was I was able to keep the console of the ESX and have the VMDP working.


With the local console you should be able to apply patches manually through the shell (I haven't done that very often; not sure how it works exactly).

I am glad to ear that some of you were able to get the DXVA working. I gave some more tests on my own and had not so bad results.

- Nvidia GT210 did not work (driver failed to load as far as I can remember)

- HD5450 did work, but I was not able to get the DXVA. The driver related to the HDMI sound was not able to load though

In order to get DXVA you have to disable the vmware GPU I guess. How have you done it? Did you simply disable the display or did you disable the whole GPU in the device manager?

I ll try to do some more test tonight; using a VM as a HTPC would be so cool. Which leads me to another related question. Have anyone tried to use a TV tuner through VMDP? Any feedback welcome.

Reply
0 Kudos
FuNK3Y
Contributor
Contributor

I was able to get the HD5450 working, even with the sound but...

- I was unable to get the DXVA working. Everytime I tried I got a green picture and the application crashed (tried with ffdshow DXVA and flash). I allready had the driver up to date (11.5 & 11.6)

- TV through 7MC did not work, because it makes use of DXVA.

- TV Tuner (Nova-T) nearly worked. It detected some channel (108 on the first time, 92 on the second time instead of the 130 channels), but I was unable to get any picture because of the DXVA bug (sound was fine though).

- Some time the picture flickered and the colors got inverted for a while. It may be related to my cheap cable

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

For a TV tuner that always works virtual or otherwise try HDhomerun.

-- David -- VMware Communities Moderator
Reply
0 Kudos
sevet
Contributor
Contributor

I'm trying to get this working for a couple of sleepless days now.

My motherboard  is Intel DP67BG which seems like the only P67 chipset motherboard with VT-d.

I got the passthrough working for SATA controller.

When I pass through a GPU,  one of my nVIdia PCIe cards - 6200 and 8400GS, I see the driver loaded ok but I have no display available in the screen settings,

The driver works only after I disable the driver of the display of VMWare.

nVidia control setting doesn't open and says no nVidia GPU connected.

When I try with a PCIe ATI card 4650HD  I get bluescreens on XP and Win7,

How do you install the VMWare WDDM 1.1 driver? I have latest ESXi and the driver from the tools is WDDM 1.0

I tried first with the two PCIe cards installed, then I found an old ATI Mach64 PCI card,

Tried with ESXi screen to the PCI card and passing the PCIe to the VM but always the same result, for my nvidia cards, driver works but I have no screens to select (detect only show the vmware screen even when the driver is disabled) and the ATI gives the BSOD.

Anyone got an Idead? I saw someone who checked combination of PCI / PCIe cards and looks like a PCI ATI7000ve with nVIdia PCIe might work

Anyone with any ideas?

Reply
0 Kudos