VMware Cloud Community
taylorjonl
Enthusiast
Enthusiast

VMDirectPath and ATI Radeon

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

814 Replies
MrPippy
Contributor
Contributor

Has anyone successfully passed through a video card on an Intel C20x/Xeon E3 machine?

With Windows 7, I've tried both 32-bit and 64-bit, different amounts of memory (always less than 2 GB), the last few Catalyst versions, and I always get a bluescreen when enabling the ATI card in Device Manager. If I disable VMware SVGA I can enable the ATI card, but it just doesn't work correctly (device starts but it won't work as a display or be recognized by the Catalyst control panel).

I tried it again tonight follwing BAM279's steps, but I get the same result.

On Linux, I tried Ubuntu 11.04 with several drivers and had no luck. Here's the relevant dmesg lines from the open-source radeon driver (fglrx was worse and just hung hard for many seconds). These errors are interesting--it seems like a legitimate PCI problem, not something that can be fixed just by installing drivers in a different order or something. In searching online, the only other systems with errors like these were high-end laptops that have switchable graphics, and people were trying to initialize the radeon while the switch was set to integrated.

[    2.930905] [drm] radeon defaulting to kernel modesetting.
[    2.930908] [drm] radeon kernel modesetting enabled.
[    2.930980] radeon 0000:03:00.0: enabling device (0000 -> 0003)
[    2.931054] radeon 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    2.932212] [drm] initializing kernel modesetting (CEDAR 0x1002:0x68E1).
[    2.932233] [drm] register mmio base: 0xB2420000
[    2.932234] [drm] register mmio size: 131072
[    2.932309] radeon 0000:03:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x0] (bogus alignment)
[    2.932325] radeon 0000:03:00.0: BAR 6: can't assign [??? 0x00000000 flags 0x0] (bogus alignment)
[    2.932327] [drm:radeon_get_bios] *ERROR* Unable to locate a BIOS ROM
[    2.932329] radeon 0000:03:00.0: Fatal error during GPU init
[    2.932331] [drm] radeon: finishing device.
[    2.932333] [TTM] Memory type 2 has not been initialized.
[    2.933355] vga_switcheroo: disabled
[    2.934000] radeon 0000:03:00.0: PCI INT A disabled
[    2.934005] radeon: probe of 0000:03:00.0 failed with error -22

I've tried everything in software to try and make this work: different PEG/PCI settings in BIOS, messing with reset methods in /etc/vmware/passthru.map, different PCIe slots, everything short of different video card/motherboard.

It just seems to me like there is some variation in either the chipset, BIOS, or video card that makes this work or not. The only way to find out is to gather info from systems where it is working. Linux is great for this--much better diagnostic/debugging info, and the radeon driver is open source.

Someone who has this working (like you, BAM279): can you install Ubuntu 11.10 in a VM, pass the video card to it, and see what happens (use the open-source driver). If it works, we can really try to figure out what's going wrong on other machines. Thanks!

My info:

M/B: ASUS P8B-E/4L

CPU: Xeon E3-1230

ESXi 5.0

Video cards tested:

- ATI Radeon HD 5450 1GB (PowerColor AX5450 1GBK3-SH)

- NVIDIA GeForce 8800GT

I also have a LSI 1068 controller passed through to a Solaris VM, but testing without it didn't change my results.

Reply
0 Kudos
aaronvm
Contributor
Contributor

New to the thread, am having the same atikmpag.sys BSOD issue as everyone else.  Most of my trial and error has been with Win7 x64, although I've also been unsuccessful at getting the Radeon card to work under an Ubuntu 11.10 VM.

My hardware:

- Shuttle SH67H3 xPC

- Intel i5-2500 Quad Core 3.30GHz

- 16GB Corsair XMS3 RAM

- ATI Radeon 5670 1GB

FWIW, other PCI devices seem to work fine through passthrough (i.e., USB controller, NIC).

In an attempt to add some value to the thread, I did find an interesting post last night on the SageTV forums:

http://forums.sagetv.com/forums/showthread.php?p=422804&postcount=24

Has anyone explored this avenue?  I haven't been able to find a way to show the status of the write-combine setting in my card, or disable it.

Reply
0 Kudos
BAM279
Contributor
Contributor

Hi MrPippy,

im downloading Ubuntu 11.10 32bit now and will install and attempt to pass through my ATI HD 3400...  I am a complete linux noob so your assistance may be needed in extracting the relevant diagnostic info you need, but I will do what I can to help.

BAM.

Reply
0 Kudos
MrPippy
Contributor
Contributor

I did a lot more poking around with lspci in both a Linux VM and in the ESXi console, and I know why the Linux radeon driver isn't working: the video card's expansion ROM isn't being passed through by ESXi

Here's the lspci output from inside a Linux VM:

$ sudo /usr/sbin/lspci -d 1002:68e1 -v
13:00.0 VGA compatible controller: ATI Technologies Inc Device 68e1 (prog-if 00 [VGA controller])
Subsystem: Hightech Information System Ltd. Device 3000
Physical Slot: 224
Flags: bus master, fast devsel, latency 64, IRQ 16
Memory at c0000000 (64-bit, prefetchable) [size=256M]
Memory at d2900000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 6400 [size=256]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting

And the lspci output using the PCI config space dumped from the ESXi console (how the video card looks to the bare metal OS):

$ /usr/sbin/lspci -F esxi_pci -d 1002:68e1 -v

01:00.0 VGA compatible controller: ATI Technologies Inc Device 68e1 (prog-if 00 [VGA controller])

     Subsystem: Hightech Information System Ltd. Device 3000

     Flags: bus master, fast devsel, latency 0, IRQ 11

     Memory at d0000000 (64-bit, prefetchable)

     Memory at fb820000 (64-bit, non-prefetchable)

     I/O ports at e000

     Expansion ROM at fb800000 [disabled]

     Capabilities: [50] Power Management version 3

     Capabilities: [58] Express Legacy Endpoint, MSI 00

     Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+

(bold mine)

The card has an expansion ROM (every video card does) and the radeon driver needs to read it in order to load, but ESXi just isn't passing it through.

I'm very curious if this is also true for people who have gotten Radeon passthrough working. If not, then what is the key variable: chipset? system BIOS? ESXi? VMware's BIOS? video card/BIOS?

Reply
0 Kudos
BAM279
Contributor
Contributor

MrPippy,

I done the various lspci terminal commands in my Ubuntu VM and the expansion ROM was missing from the output on mine as well.

I havent been able to find or install the download for the open source radeon drivers as I have no idea what exactly im looking for due to my lack of linux knowledge.

BAM.

Reply
0 Kudos
meimeiriver
Enthusiast
Enthusiast

Has the >2GB problem been solved in ESXi 5? I'd love to hang my GTX 580 in a VM!

Reply
0 Kudos
MrPippy
Contributor
Contributor

BAM,

Thanks for trying it out. The open-source radeon driver is included with Ubuntu and it will automatically load. Run 'dmesg' in the terminal, and near the bottom you should see some radeon and drm lines like '*ERROR* Unable to locate a BIOS ROM' and the other ones I posted

Reply
0 Kudos
BAM279
Contributor
Contributor

MrPippy,

yep, confirmed - dmesg outputs those radeon bios rom missing errors Smiley Sad

BAM.

Reply
0 Kudos
BAM279
Contributor
Contributor

Meimeiriver,

There is a solution to successfully enable >2GB vRam by using an advanced config option called "pcihole" - you should look back through the thread for the specifics.

BAM.

Reply
0 Kudos
meimeiriver
Enthusiast
Enthusiast

Thanks! Appreciate it!

We live in exciting days. Smiley Happy Long live Vmware!

Reply
0 Kudos
WhipeOut
Enthusiast
Enthusiast

I haven't read the whole thread here, but I've done something very similar. And it looks like you beat me to it as well.

You can read more about it on my blog at www.v12n.com. Look for "ESX and GPU utilization"

Reply
0 Kudos
DennisMaa
Contributor
Contributor

I have successfully managed to get my test machine to work, here is a list of the hardware and installation process i used.

Hardware:
- Supermicro X8ST3
- Intel i7 960 @ 3.20GHz
- 24GB memory
- Soundblaster Audigy 4 (directpath)
- XFX Radeon HD 6870 1GB (directpath)


Installation process:
- install windows 7 x64 with 2GB memory plus windows update
- Added Soundblaster (installed flash)
- Added both devices for the ATI card
  - started upp the machine and rebooted right away, because of new hardware.
  - installed Catalyst 11-11 win7 64
  - rebooted machine again (monitor started to work)
- Added 2GB of memory to a total of 4GB and got the error:
    Module MemSched power on failed.
    An error occurred while parsing scheduler-specific configuration parameters.
    Invalid memory setting for FPT: memory reservation (sched.mem.min) should be equal to memsize(4096)

- Changed sched.mem.min to 4096 and now it works with 4GB of memory.

I used ESXi 5.0.


Let me know if you want any more details.

Reply
0 Kudos
jhonnypolak
Contributor
Contributor

Hi Guys,

Been reading this thread for a while now. Very interesting and thank you to all those that have put in the time for testing.

I've tried to recently set this up as well. My specs are below:

Intel i7 2600

Shuttle SH67H3 (Intel H67 chipset) (VT-d supported)

Dell Radeon HD3450 256MB PCIe (DVI only)

ESX i 5.0.0 (build: 504890) (latest)

The VM that i'm trying to pass through the video card is:

Windows 7 Pro 64bit

4GB RAM with applied fixes mentioned in this thread (tried 2GB made no difference)

What I found is:

Video driver for VMWare SVGA uninstalled

HD3450 enabled in Device Manager

Catalyst Control Center doesn't detect the card

Can't output to the monitor attached to the HD3450

Video driver for VMWare SVGA installed

HD3450 enabled in Device Manager

System BSOD: atikmpag.sys

I've tried installing VMWare tools from ESXi 4.1 even though I'm running ESXi 5 (for testing purposes) however taht didn't make a difference.

I tried Catalyst Drivers, 11_5, 11_6, 11_11, all get BSOD.


I think this is possibly something to do with the VMWare SVGA driver and the driver of the HD3450.

I think there might be discrepancies between the way people create their VM's.

Can someone who has the 3450 or other radeon card working post the details of their VM ?

(edit VM settings and go to the Options tab)

ESXi version:

Virtual Machine Version: (6/7/8)

General - Disable acceleration (checked / unchecked)

General - Enable logging (checked / unchecked)

General - Debugging and Statistics (which option)

CPUID Mask: (Hide / Expose / Keep current)

Memory/CPU Hotplug - Memory Hot Add (Disable / Enable)

Memory/CPU Hotplug - CPU Hot Plug (Disable / Enable hot add only / Enable hot add and remove)

Boot Options - Firmware: BIOS / EFI

Fibre Channel NPIV (checked / unchecked to Temporarily Disable NPIV)

CPU/MMU Virtualization (Automatic / Use software / Use Intel VT-x and software for MMU / Use Intel VT-x and Intel EPT for MMU)

Resources tab:

Memory - Reserve all guest memory (checked / unchecked)

Hardware tab:

Memory - Memory Size:

CPUs - Number of Virtual sockets:

CPUs - Number of cores per socket:

Video card - (Auto-detect / Custom)

Video card - Number of displays:

Video card - Total video memory:

Video card - 3D graphics - Enable 3D support (checked / unchecked)

VMCI device - enable VMCI between VMs (checked / unchecked)

Also if you can post:

- a copy of your vmx definition file for the VM you have a Radeon working in, that would help a lot too.

- Radeon driver version (also if its from ATI or vendor)

- VMWare tools you're using

- Driver version of VMWare tools SVGA adapter

NOTE: If you're using ESXi 4.1 or you might not have all these options.

Reply
0 Kudos
joiegils
Contributor
Contributor

just want to let you know my setup

motherbard=intel dx58so2

cpu=intel i7 950

memory=12gb

vmware 5 instal on 500gb hdd and nexenta is also instaled on that disk

nexenta machine 5gb memory get the 3x1tb disks in rdm and then i use iscsi for the machines on vmware

windows machine 1 = ati 5450 in passthroug and usb controler 1 in passthroug also the machine has only 1gb memory and i run just xbmc on that machine. the usb controler has usb hub and then i plug evrything in the hub to get it to work in the machine i have usb sound adapter with only jack out because this machine brodcast only vga to my chitcen monitor and i control it with pc remote cheap remote but works.

windows machine 2 = ati 5450 in passthroug and usb controler 2 in passthroug this machine also have just 1 gb ram and also runs xbmc and i plan to use the hdmi on that card to my machine in bedrom with hdmi over cat-6, have tried the hdmi and that works and sound also this machine also have usb hub and the remote conects to the hub.

windows server = this machine has 2gb ram and is just a fileserver and torrent machine and gets 2x2tb disks throug rdm with the disk

linux machine = this machine gets wintv hvr-1110 throug passthroug and is just a vdr machine to stream to my house and to record it gut just 1gb memory

i have only once gut problem with the graphics cards in passthroug but after the ati drivers where put on the machine everything works,

Untitled.png

sorry for my english

Reply
0 Kudos
jhonnypolak
Contributor
Contributor

joiegils can you tell me what version of ESXi you're running ? build number ?

I've done a bit more testing in my environment and found that when i create a VM version 7 rather than 8, I can get further.

I.e. card installs, driver installs, multiple displays show up in desktop properties, but when I extend to a display connected to the Radeon i get a BSOD immediately.

I also tried ESXi 4.1 U1 build: 348481, however that also gives me BSODs.

On a side note, just for curiosity's sake i installed XenServer 6.0, just because I wanted to make sure that my hardware was capable of passingthrough the GPU and guess what ? ... it works flawlessly.

I was able to passthrough the Radeon HD3450 but not the Intel HD2000.

An interesting note about that is that when XenServer enables the HD3450 it automatically disables the "Virtual" vga driver. Which i think is the problem right now that a lot of us are experiencing with ESXi, since it doesn't disable the SVGA 2 adapter when it enables the vt-d'd gpu.

Reply
0 Kudos
joiegils
Contributor
Contributor

i´m runnin exsi 5.0 build 504890 after uppgrade because of iscsi slow boot 

but i got it working also on exsi 4.1

and i create a VM version 8 for my machines

Reply
0 Kudos
aaronvm
Contributor
Contributor

That is very interesting.  I also have a Shuttle SH67H3, but hadn't tried XenServer.  This seems to confirm that the motherboard/chipset is not at fault.  (I was wondering about this, since it seems a lot of people have reported success with SuperMicro.)

The card I can't get working is the XFX Radeon HD 5670.  I've tried both Windows 7 64-bit, and the latest Ubuntu 64-bit.

Curious if you've tried a 32-bit Windows install?

I like your theory about the SVGA driver, maybe if any VMware engineers are listening they can comment on a possible workaround.

Reply
0 Kudos
jhonnypolak
Contributor
Contributor

I haven't tried a 32bit VM. It might be worht a shot

I've got XenServer 6.0 installed now.

I've got Windows 7 pro 64bit installed on it.

The Radeon HD 3450 works without issues.

I could also passthrough USB 2.0 ports to the VM

USB 3.0 ports even though i could pass them through to the VM and it detected the device correctly in Device Manager and drivers installed successfully too.... any device I plug into the USB 3.0 ports doesn't work right.

Also passing through the Intel HD Audio doesn't get detected in the VM 😕

I wonder if a USB Audio device would work right ? I read somewhere that XenServer supports only mouse/keyboard with usb passthrough (which isn't officially supported anyway)

I tested the USB 2.0 also by plugging in a USB memory stick, that worked, no problems Smiley Happy

So far it seems to me that neither solution is perfect for the SHuttle. I really do think these are software limitations though.

I'd be inclined to test a bit more on VMware as its easier to do.

Xen usb / pci passthrough is all command line based as its not officially supported. It takes a while to get around the CLI.

Please let me know if you get any further with testing your 32bit VM

Reply
0 Kudos
jhonnypolak
Contributor
Contributor

A quick update.

I managed to get Intel HDA audio working.

I found if you do this it works:

Go to shuttle website and download drivers. Install them.

Shut down the VM.

Add Intel HDA PCI device to the VM.

Restart the XenServer.

Turn on the VM, it will detect the Intel HDA and install required drivers automatically.

The audio works perfectly fine in Windows Media Player. Smiley Happy

Nice .... now to install OSX on this bad boy Smiley Wink

Reply
0 Kudos
derickso
Contributor
Contributor

I've got a couple questions, I am in the process of building an all in one setup with ESXi and an Open Solaris ZFS file server VM, and I ran into this thread which got me wondering if I could combine my existing Windows 7 Media Center PC with this new box in a VM.  I am fine with buying a new ATI card if needed to make things work, but a few things I have questions about:

-Can I pass through USB devices like my IR receiver, keyboard, mouse to the VM?

-Has anyone gotten HDCP working properly to watch 'protected' content?

-Has anyone been able to watch a blu-ray from the VM?

-Is video or audio stuttery at all when watching 1080P HD content?

Thanks,

David

Reply
0 Kudos