VMware Cloud Community
NotYetRated
Contributor
Contributor

ESXi 5.5, Passthrough Video Card, Haswell Build

I am currently having difficulties with ESXi 5.5, in passing a video card on to a Windows 7 install.

Components of Note:

Motherboard - http://www.newegg.com/Product/Product.aspx?Item=N82E16813157409

Processor - Xeon e3-1245v3 http://www.newegg.com/Product/Product.aspx?Item=N82E16819116909

RAM - 4x8GB for 32Gb total

Intel 520 180GB SSD for Datastore

ATI Radeon 6450 Graphics Card

I have installed Windows 7 64 bit, passed on a USB controller, where I have a working mouse/keyboard setup. Installed all windows updates, then rebooted and enabled the Radeon 6450 passthrough. It seems to pass through okay, I install drivers, but the system does not recognize any monitor plugged in to the graphics card. It appears to continue to only use the virtualized display adapter.

Also, I can not get the system to boot with any more than 2 gigs of RAM when the video card is passed through.

I have also tried passing through the integrated video chip on the Haswell processor/board, but I get the same results.

If I install VMware tools with the video card passed through, I get a BSOD.

Any ideas? Below is a pic of my passthrough options from this hardware combo.

http://i.imgur.com/NQ1zVc5.png

40 Replies
zyonov
VMware Employee
VMware Employee

I have Haswell build at home using motherboard from Asrock - Z87m extreme4. I experienced exactly the same issues as you and this thread helped me a lot to solve them: Re: VMDirectPath and ATI Radeon

This is my answer to your questions, hopefully they will solve your problem.

1. To get VM working with passthrough and more than 2 Gigs of RAM, edit your vmx file and add at the end of it these 2 lines:

pciHole.start = “1200″

pciHole.end = “2200″

2. To make your ATI card working properly without BSOD, you should disable the integrated GPU from the motherboard BIOS. You should stop seeing "Haswell Integrated Graphics Controller" as available for passthrough in your ESX host.

Then power on the VM, install OS, VMware tools, drivers etc. and see if it's working.

Hope that helps.

Z

Heinrich_Rebehn
Contributor
Contributor

Hi zyonov,

As i am planning to build a similar setup, i want  to ask you about your experience:

What is your overall experience with your board?

What CPU are you using?

Is the onboard LAN supported by ESXi?

TIA

H

Reply
0 Kudos
wenlez
Contributor
Contributor

Have you try to passthrough your onboard SATA controller?   Or passthrough your HD4600 iGPU to a vM and see if it can output to HDMI?   I'm really interested to build an Intel Haswell ESXI host and use it as a HTPC as well.

Reply
0 Kudos
zyonov
VMware Employee
VMware Employee

I'm using my internal SATA controller as datastore for my VMs, so I haven't tried to passthrough SATA, otherwise I will loose this datastore. I've seen posts where people are saying that they can't passthrough onboard SATA controller nor the integrated HD4600 iGPU.

So far I managed to passthrough my onboard sound card, one of the USB controllers and my ATI Radeon 7770 video card to win8 VM. My experience so far:

1. Onboard sound card has poor performance, with choppy metallic  sound (happening from time to time)

2. No problems with my discrete graphic card. I haven't tried the sound output from the ATI card, as I don't need it. I suppose it will be fine, as I don't see any issues with graphic performance under that win8 VM. No problem playing games too.

3. No problems with USB controller.

4. Have connected external, USB sound card, to that VM which works flawlessly.

I suppose you can build ESXi host and have a HTPC VM, but for that purpose you need discrete video card. Probably you will achieve smaller form factor if you just build HTPC with ITX motherboard, rather then building server grade ESXi host.   

Reply
0 Kudos
wenlez
Contributor
Contributor

would you try to passthrough the Intel HD4600 iGPU to a windows VM see if it's detected? i really want to use it for a HTPC set up.

Reply
0 Kudos
som12
Contributor
Contributor

Great thread! I've been having issues for 2 weeks and any help would be appreciated


Specs:

ESXi 5.5

Asrock - Z87m extreme4

i5 4570S

Asus Radeon 7750


I've passed through my Asus Radeon 7750 to a Win7 64bit (EFI). Vmware tools installed. The drivers install without any errors. The device manager detects the card as a Asus Radeon 7750, as well as the vmware display device. Using the vmware console, I can only see one screen in the Windows display settings. What am I doing wrong?


For those who has gotten gpu passthrough to work - what ESXi version are you running? What else should I have done to make this work?

Reply
0 Kudos
downh1ll
Contributor
Contributor

I have exactly the same problem with an MSI motherboard (MSI Z77A-G43) and a Radeon R9 270X on ESXi 5.5 update 1. Side-note: It works great in competing hypervisors, but I'd prefer ESXi as I'm familiar with VMware's way of things and it offers broader support for guest operating systems. I'd *really* love to make my computer a dedicated server and retain the gaming side of it.

Could anybody shed some light on this problem of only having 1 screen show up even though the physical graphics card has drivers loaded and seems to start without a problem otherwise?

EDIT: I solved my own problem and I will be responding to this with clear notes on how I fixed it because this thread shows up in Google's results quite often.

Reply
0 Kudos
zyonov
VMware Employee
VMware Employee

I have separate monitor connected with my discrete graphic card, that I passthrough to the VM. Console is using virtual graphic adapter not the discrete one.

Reply
0 Kudos
downh1ll
Contributor
Contributor

In order to get my Radeon R9 270X working with PCI passthrough on ESXi 5.5 Update 1, running on my MSI Z77A-G43 motherboard and Intel i5-3550, I did the following:

#1: Updated my BIOS to the latest version (v2.13 at the time of writing).

#2: Disabled my onboard sound card. Unfortunately this means I need to dedicate another PCI slot to a sound card to pass through. This problem seems to be limited to ESXi as a competing product that offers easy "VGA passthrough" can do both my Radeon and onboard sound at the same time.

#3: It seems that I MUST create the VM for which I want to pass a graphics card to WITHOUT the PCI device (graphics card) assigned in the VM settings.

#4: It also seems that I MUST install VMware Tools into the guest VM which I want to pass a graphics card to BEFORE adding the PCI device (graphics card) to the VM in the settings dialog.

#5 Passing through my onboard USB (bad idea, folks!***) doesn't work in the guest. I get "Device cannot start. (code 10)" in device manager. I haven't discovered a way to remedy this. Again, I need another add-on device just to pass through to a guest VM. I've now ran out of PCI/PCIe slots. C'mon, VMware!

***if booting from USB this is a very bad idea! You "can't undo it without significant effort" as the passthrough dialog in vSphere Client states. Besides, it slows the system down significantly and makes it impossible for host configuration changes to persist across reboots (including changing devices assigned for PCI passthrough).

Apologies if I have forgotten anything in particular. I'll try to look back later for questions. I can't guarantee this will work for you, seeing the spotty feedback on PCI passthrough with graphics in particular, but if you're using the same motherboard it should get you going.

Best of luck!

Reply
0 Kudos
JohnMille78
Contributor
Contributor

Hi everyone,

I have something very similar to downh1ll :

- Asus rampage IV black edition

- 16G RAM

- 2x R9 290X ATI

- 2x SSD 250G

It took me all this night to figure out the correct BIOS Settings to have esxi 5.5. working properly and stable ( I was having non-explained crashs ) and now be the happy user of my workstation in a VM with GPU Passthrough working like a charm.

My problem today is double :

- Assume CG1 is the one on which the system boots, after boot, I cannot have a VM using it for passthru

- CG2 is successfully used by ESXi 5.5. the first time the guest (Win 8.1 N x64) but if I shutdown the guest and reboot the vm, the passthru doesnt work anymore and the software renderer takes over. The only workaround till now is to reboot the host.

Has anyone any idea to solve these 2 issues ?

PS : let me know if you are interested in the DUMP of the BIOS settings.

Reply
0 Kudos
laptopfreek0
Contributor
Contributor

I can help you with the second issue: In order to have the card not crash on reboot you have to safely eject the VGA card before you reboot it: here is a tutorial on how to set a reboot script to safely eject the card before you reboot: http://blog.ktz.me/?p=219

*Edit while the above fix does work, it requires you to re-add the GPU every time before you start the VM (What a pain in the butt).

But on the first issue, I appear to have the exact same issue. Using 290x for primary and 290 for secondary, think the issue might be related to how esxi uses the primary video card to display the splash screen.

Reply
0 Kudos
devianceluka
Contributor
Contributor

Hello.

I also have R9 290 and can't seem to passthrough it successfuly to a Win8.1 64bit VM without Error 43. Can you please explain what you did to get it working? I have already tried downh1ll's suggestion to not install Windows with passthrough and first install VM-tools then passtrough the GPU. It's the exact same thing (error 43). With Radeon 6970 it works flawlessly without doing anything "special".

Thanks!

Reply
0 Kudos
JohnMille78
Contributor
Contributor

Hi,

I have not done anything specific to have it working. As told, the primary card is not working as it is the one used by the system to get started.

First step was to create the Win 8.1 VM with no passthrough.

Then I added the graphical card and USB as passthrough and that was just working fine.

With the Rampage IV I had to enable VT-d and set speeds to GEN2 instead of GEN3 because on GEN3 after some times, I had the entire machine crashing.

In GEN2 it was more stable.

Is error 43 an ESXi code or Windows code 43 ?

Reply
0 Kudos
devianceluka
Contributor
Contributor

I have enabled VT-d and I actually have 3x R9 290. I first install Win8.1 inside VM (1x EFI VM, 1x legacy VM) without passthrough. Then second I install VMtools, then third shutdown and passthrough R9 290+HDMI and install drivers. I cant get no monitor output because the R9 290 has Error 43+exclamation mark inside 'device manager'. Whichever of the 3 cards I passthrough, situation described is the same. ESXi is patched to latest and I didn't passthrough USB just yet. If at any point replace R9 290 with 6970 at whichever PCIE port (also if it is the only one in first port!) it works flawlessly as it should! With R9 290 I also tried (2GB,4GB,5GB RAM; pciholestart 1200, pcihole.end 2200, pcipassthru0+1.msienabled false,.......). What should I do?

JohnMille78
Contributor
Contributor

The only time I had an error I could "read" / "see" the system asked me to up the pciHole.start to 2100. That's the only parameter I changed.

Other than that on the BIOS side, I was using UEFI and secure boot disabled.

I think this value depends on the memory of the Graphic Card, but I am not sure.

This is it really, I had the ESXi setup via cobbler, then just uploaded the ISO to datastore, create the VM. I have a Win 8.1 Pro N. Maybe the version matters ?


In the between I have done an upgrade of my BIOS so I have the config flushed but, that's it basically. I think it really is just 1 setting that makes a difference, from the issues I had, on the hardware side.

Reply
0 Kudos
devianceluka
Contributor
Contributor

What does "CG" mean?

Also by 2100 you mean pciHole start or end? Can I ask you where you came up with that number? Because I have searched on google for an explanation on this command and didn't find anything so I don't really know what it means so I can play with it.

Reply
0 Kudos
JohnMille78
Contributor
Contributor

pciHole.start is the only I have changed : not touched to pciHole.end

It is VMWare itself that gave the value in the logs ("events") of the virtual machine.

Reply
0 Kudos
devianceluka
Contributor
Contributor

Again what does CG mean?

Also how much RAM did you give to this VM? Is it EFI or legacy?

Reply
0 Kudos
JohnMille78
Contributor
Contributor

I edited my original post to have future people to understand. I meant "GC" -> Graphical Card

I had 16Go on the host, 8G on the VM 100% dedicated and so were the vCPU (4 vCPUs)

Reply
0 Kudos