VMDirectPath and ATI Radeon

[taylorjonl]

I am using VMware ESXi and I am trying to setup a guest that is Windows 7 that will have an ATI Radeon video card passed through to it.  I actually had this working on a previous system but I had to reinstall.  Now when I do this the guest fails to start and I get the following:

Error message from localhost.XXXXXXXXXXX:
PCIPassthru 004:00.0: Guest tried to (null)map
32 device pages (with base address of 0xb5d20)
to a range occupied by main memory. This is
outside of the PCI Hole. Add pciHole.start =
"2909" to the configuration file and then power
on the VM.
error
12/23/2010 1:04:36 PM
media
User

When I do as it asks, the guest now starts but gets an immediate BSOD concerning memory management.  Any ideas on why this is occuring and why it worked at one point but now it fails?

[rmathis1984]

Both have the 8111E NIC's which do work. Stability is in questions its been a hit or miss for me. I have that same NIC in several machines over the past few years. Sometimes its rock solid sometimes it's not. As long as you use the lastest updates on 5.0u1 you shouldnt see much of a problem unless you start dropping VLAN's on then you might. Use a cheap Intel NIC its worth the $20 to $30... Ignore the possible headaches.

For the Max speed on a NAS unit I have FreeNAS running and see 280MBs copys inside the host and 110MB's outside on 1Gbs switch. Saturated links arent much of a problem anymore. If you use FreeNAS avoid ZFS unless you want to dump at least 4Gb memory of you wont see the functions of ZFS in use or it simply wont cache.

I can't speak for the onboard SATA but I havent run into any controllers in 3 years that havent worked out of the box on desktop hardware.

"Is it possible to share the DVI, VGA, HDMI between VM's at the same time" = Nope. You'll purple screen the host as the VM's will be fighting for the cards memory. Maybe with some crazzy codeing you could but dont execpt to see that on cards under 1k. A cheap card will do look up 2 or 3 posts...

For putting the machine to "Sleep" and using it. I havent tried it but I would assume it might not work out so well for stability of the host. Imagine having your brain shatched for about 20 minutes then handed back... My guess is the VM will notice activity on the card and start up. I would shut down the VM. I currently have a Win7 box and a Linux box sharing as long as there not on at the same time there isnt a problem at all. Makes for a fun time trying to change machines glad to have a tablet to power on the desktops lol.

As long as the CPU supports IOMMU "AMD guys chip in dont think thats correct" those boards should be good. Worst case call the mobo manufacture ask if it supports passthrough.

[derickso]

Not sure about the SATA ports, but I'm pretty sure you cannot pass a single one of those through, it generally has to be the whole chip.

You cannot individually pass through a single output to different VMs, you have to pass through the whole card. If you need a cheap solution look at the 7750 I linked, it has 3 outputs, DisplayPort/HDMI/DVI-D, you may have to get adapters if your monitors only take DVI.

If you stop a VM that was using a passed through card you can start another that uses it, but you can't have two running simultaneously with the same hardware.

[bwiklander]

And about the USB.

The Gigabyte 990FXA-UD3 has one usb 2.0, and one 3.0 so you have 2-diffrent usb controllers.

And i also think that passing trough a single SATA port isnt going to work..

Im running a NAS inside a VM and is passing trough my whole SATA controller. and it works great, even using ZFS. Altough i have run into trouble with Freenas only being able to use one CPU-Core, otherwise its giving me "IRQ throttle error stuff"

I aswell think this is a great forum. one suggestion though. If you have as many questions as you had in your post, try to sort the like number wise.

1. usb

2. sata

etc.

Cheers!

2012/12/3 derickso <communities-emailer@vmware.com<mailto:communities-emailer@vmware.com>>

VMware Communities<http://communities.vmware.com/index.jspa>

VMDirectPath and ATI Radeon

reply from derickso<http://communities.vmware.com/people/derickso> in VMware ESXi™ 4 - View the full discussion<http://communities.vmware.com/message/2157416#2157416

[seertenedos]

derickso wrote:

Not sure about the SATA ports, but I'm pretty sure you cannot pass a single one of those through, it generally has to be the whole chip.

You cannot individually pass through a single output to different VMs, you have to pass through the whole card.  If you need a cheap solution look at the 7750 I linked, it has 3 outputs, DisplayPort/HDMI/DVI-D, you may have to get adapters if your monitors only take DVI.

If you stop a VM that was using a passed through card you can start another that uses it, but you can't have two running simultaneously with the same hardware.

Ok so if i need to pass though the whole chip that is fine but do any of the on board controllers work out of the box with esxi or does anyone know of a cheap pcie controller where a can put a disk or 2 just for esxi?  I need to get about 12 disks into a VM so i am hopinh to pass though the bigger controllers on board if possible and maybe a cheap sata card and expander as the 16 port cards cost a small fortune due to the raid that comes with them that i don't need.

Thanks for the tip onthe 7750 i will look into that as i already have displayport converters.

Can you just suspend a VM using passthorugh or do you need to shut it down? I like the idea of just suspending them so there is no bootup times etc.

[KRA77]

You can't pass on a single port, but you can set up RDM (raw device mapping?) google it.

I have passed on a 2TB disk to a VM with RDM. On the same controller I have a SSD for ESXi, then I have a ASUS PIKE controller passed on to Openindiana ZFS system.

[bwiklander]

1."You can't pass on a single port, but you can set up RDM (raw device mapping?) google it."

sorry, didn't know that,

Is RDM the same thing, do oyu get full access to the disk?,

can you for example put it to sleep and such like "advanced IO functions?"

2. About the question about shutting down VM's to use the VGA card on two diffrent VM's

I think you have to do that, i Think ESXI has a built in function that prevents VM's that has the same PCI-passtrough device to be "ON" at the same time,

But cant you like put the computer to Hibernate or such, or Suspend to disk, and then power it off?

Otherwise if you have the VM's on a SSD you wont have that much of a boot time. i think my testning machine boots in 20sec or so

2012/12/3 KRA77 <communities-emailer@vmware.com<mailto:communities-emailer@vmware.com>>

VMware Communities<http://communities.vmware.com/index.jspa>

VMDirectPath and ATI Radeon

reply from KRA77<http://communities.vmware.com/people/KRA77> in VMware ESXi™ 4 - View the full discussion<http://communities.vmware.com/message/2157368#2157368

[derickso]

Google M1015 if you need a cheap host bus adapter to pass through, I'm using one I paid about 90 bucks for and passing it through along with 8 drives connected to it, cost was 90 bucks or so used.

You cannot suspend (from ESXi's point of view) VMs using passed through hardware. You could suspend the OS itself but that won't accomplish much.

[KRA77]

I'm not sure how powersaving options etc work with RDM, afterall it's still a virtual controller, only the VM writes to the disk as if its a normal disk (so you can take it out and read it in another computer)

More info:

http://vm-help.com/esx40i/SATA_RDMs.php

[seertenedos]

Thanks all. I think you have almost answered all my questions. The only one i have left is to do with the gigabye motherboard and if the on board sata and network it recognised by esxi or if i need to use cards for that.

I am looking at the M1015 for passing most of my disks though to my Freenas VM.

[rmathis1984]

@bwiklander

//

1. "You can't pass on a single port, but you can set up RDM (raw device mapping?) google it."

sorry, didnt know that,

Is RDM the same thing, do oyu get full access to the disk?,

can you for example put it to sleep and such like advanced IO functions?"

\\

Nope RDM is not the same. RDM is direct access to a SAN/NAS volume. Where in this case he would need to pass the local controller to the VM which isnt the same as its a local storage disk he's passing.

[bwiklander]

@ rmathis1984<http://communities.vmware.com/people/rmathis1984> ,

Ok, thx for the clearup!

About the Gigabyte Board.

i have the Gigabyte 990FXA-UD3,

and the onboard NIC, is working great in ESXI 5 and is recognized.

The Onboard Sata, was also recognised and did work great, i am passing this trough to my FreeNAS VM right now. and am using a cheap PCI device for my ssd disks in ESXI.

2012/12/3 seertenedos <communities-emailer@vmware.com<mailto:communities-emailer@vmware.com>>

VMware Communities<http://communities.vmware.com/index.jspa>

VMDirectPath and ATI Radeon

reply from seertenedos<http://communities.vmware.com/people/seertenedos> in VMware ESXi™ 4 - View the full discussion<http://communities.vmware.com/message/2157493#2157493

[jmmarton1]

smanet wrote:

MSI Z77A-G41 Intel Z77 Chipset

Intel Core i5-3570 (not k)

Geil Evo Corsa PC3-13600 32 GB (Quad-kit)

XFX AMD Radeon HD 6450 GDDR3 1 GB (I will buy another one, as I want to dedicate it to Hyper-v inside ESXi. I know I'm crazy )

Sorry to hijack this thread and also respond to an older post, but I'm curious how you go that motherboard to work.  I have the same MSI motherboard with the latest BIOS (2.70), and the Intel Core i7-3770 (not k) CPU.  I'm just using the integrated Intel video.  I've tried installing ESXi 5.1 (79973) and ESXi 5.0U1 (623860).  In both cases I keep getting the same error during the install that "No network adapters were detected."  Some Googling seems to indicate the Realtek 8111E on this board should work, at least with 5.0, so I assume also 5.1.  Is there a specific BIOS setting you had to enable to make yours work?  Or did you just install a separate NIC?

Joe

[jmmarton1]

jmmarton1 wrote:

Sorry to hijack this thread and also respond to an older post, but I'm curious how you go that motherboard to work.  I have the same MSI motherboard with the latest BIOS (2.70), and the Intel Core i7-3770 (not k) CPU.  I'm just using the integrated Intel video.  I've tried installing ESXi 5.1 (79973) and ESXi 5.0U1 (623860).  In both cases I keep getting the same error during the install that "No network adapters were detected."  Some Googling seems to indicate the Realtek 8111E on this board should work, at least with 5.0, so I assume also 5.1.  Is there a specific BIOS setting you had to enable to make yours work?  Or did you just install a separate NIC?

As a followup, I looked at the vmkernel.log file and saw that the r8168 module was loaded successfully, which from what I've read is how ESXi detected the 8111E NIC.  But soon after loading the module, this error appears in the log.

ALERT: Util: 1203: Failed trying to get a valid VMKernel MAC address: Not found. Various vmkernel subsystems will provide lower quality of service. Please add/configure VMKernel NICs.

I'm really curious now how to get this board to work properly other than by adding a separate NIC.

Joe

[seertenedos]

Any tips for usb passthrough ? Tried passing through whole controller but win 7 does not recognise it. Video passthrough is working well. Trying to get mouse and keyboards now. Running 5.0 update 1

[technobro1]

HI, i notice , i need to reboot twice to make my pastrough work ,

---------------------------------------------------

Different question

Anyone ever try 2 VM (win7) , 2 GPU, as in 2 gaming  machine.

[roancea]

Yes, have 2 vm's (among many others) running right now on a test box that both output radeon 6 series cards and usb controllers to have 2 distinct desktops that will happily run mostly the same as if they were standalone systems. 

[seertenedos]

Got my system up and running now. Thanks for all the help.

[derickso]

Add it to the Google doc if you haven't already! Thanks!

[x1nick]

I have bought the following and setup of ESXi was pretty flawless

Gigabyte GA-990FXA-UD3

AMD FX8150

Corsair 32GB DDR3

Asus HD6450

I  noticed a few posts back someone mentioned they managed to pass through  one of the graphics cards which was used for the ESXi console. I tried  doing this with the above setup but it crashes on 'cnic_register loaded  successfully'

Plan on testing a little more today as and when I get some free time, but this to me suggests you need a console graphics card?

[mindless728]

You basically have the same setup as myself

same mother board

I have the FX-8120

32GB DDR ram

I do have a console gpu, though ffrom what I have been reading through this thread you don't need one