ESXi 5.5, Passthrough Video Card, Haswell Build

[NotYetRated]

I am currently having difficulties with ESXi 5.5, in passing a video card on to a Windows 7 install.

Components of Note:

Motherboard - http://www.newegg.com/Product/Product.aspx?Item=N82E16813157409

Processor - Xeon e3-1245v3 http://www.newegg.com/Product/Product.aspx?Item=N82E16819116909

RAM - 4x8GB for 32Gb total

Intel 520 180GB SSD for Datastore

ATI Radeon 6450 Graphics Card

I have installed Windows 7 64 bit, passed on a USB controller, where I have a working mouse/keyboard setup. Installed all windows updates, then rebooted and enabled the Radeon 6450 passthrough. It seems to pass through okay, I install drivers, but the system does not recognize any monitor plugged in to the graphics card. It appears to continue to only use the virtualized display adapter.

Also, I can not get the system to boot with any more than 2 gigs of RAM when the video card is passed through.

I have also tried passing through the integrated video chip on the Haswell processor/board, but I get the same results.

If I install VMware tools with the video card passed through, I get a BSOD.

Any ideas? Below is a pic of my passthrough options from this hardware combo.

[devianceluka]

I still can't understand how is it working for you and not for me... I'm not a rookie in ESXi usage but just cannot understand. Are you using ESXi 5.5 build 1892794? Did you have to change anything in BIOS to let passthrough work?

[downh1ll]

First, I apologize if I'm leading you on a wild goose chase.

Just off the top of my head:

Is there an updated BIOS for your motherboard?

Can you test disabling nearly all of your onboard features to see if it's a conflict/BIOS bug? I've seen onboard sound mentioned various times on the forum, and it was indeed the culprit for me, in tandem with an outdated BIOS.

I would also suggest checking your driver download and making sure it installed correctly/wasn't corrupted in transit.

The fact that the other card works makes me wonder if the 290 card works on bare metal Windows. Can you try that?

Again, sorry if nothing helps or if you've done all of that already. I realize these are pretty "big" steps, but you hinted that you've done what you saw was available in ESXi, so I'd start looking elsewhere for your conflict.

Hope that helps.

-downh1ll

[Docop]

Hi

I saw Zyonov post's in regards of the Radeon 7750 into a passthrough mode over ESXI. So basically, I would like to confim the process of the build for an Asrock Z87m Extreme4

In 5.5 do you just disable the mainboard graphic and select PCI. And disable also the multigpu.  (did you leaved hd audio ON for hdmi and the hd audio sound card)

Do the passthroug of the 7750 card.

reboot,

Add it to your vm

edit vmx for the pci.hole 1200 , 2200...

boot vm

install the Radeon/catalys driver..

Did you disable the VMVGA driver into the console or as soon the Catalys loaded up

it's ok to go ? Or just don't install the VMtools vga driver ?

I try with an old Asus eax550, i can pass pretty easily, it was detected into my Win7 64 VM. I install the driver, but never show up, always into the VGASave mode. So by having this : was it because it don't work ? Obviously, no picture on the monitor screen...

Thanks in advance

[zyonov]

As far as I remember this is what I have selected in the BIOS, sorry but this machine is running 24x7 and I haven't rebooted it for a long time.

1. Only PCI graphic adapter is selected, multigpu is disabled. I didn't manage to passthrough discrete GPU with onboard GPU enabled.

2. I haven't disabled integrated audio. In my post I mentioned that I was able to passthrough the onboard sound card to the same VM, although the performance of this soundcard is poor with metallic sound.

3. I'm installing VMware tools before Catalyst.

4. I have monitor connected with my passthrough VGA card. After installing VMware tools and the Catalyst Center, windows is detecting extended display, one on the VM console and another on the external display.

5. After this I can safely disable the VMVGA adapter from Windows control panel and disabling extended display mode in windows. 

[masterameel]

I'm curious about Remote Desktop Gaming.

For those who managed to get GPU to work:

1) do you notice any performance loss in GPU?

2) has anyone tried 3D gaming in Windows with GPU passthrough?

3) If yes above, which remote desktop solution (RDP, vnc, splashtop or other) are you using to maximise performance?

I'm assuming a good bandwidth (gigabit connection) for remote desktop gaming.

I myself have the following setup:

cpu: intel i5 4670

gpu: r9 280x

ram: 16GB

HDD: 250GB ssd

GPU is passed through to Win 8.1 VDI (i didn't seem to have any issues passing through the GPU), with vmware tools installed on Win8.1

I notice SIGNIFICANT performance loss whether I use splashtop or RDP (remotefx). RDP seems to work better, but I can't go fullscreen. Still trying to find a decent solution for remote desktop gaming. Any ideas welcome.

[Timmey0206]

Hello,

i have also some problems with gpu passthrough

Maybe someone can help me

My system:

AsRock Z97E-ITX/AC

Intel Xeon E3-1245v3

16GB Kingston DDR3 RAM

MSI R9 290 4GB OC

Samsung 256GB SSD and two 3,5" SATA drives

I installed ESXi 5.5 (costumize iso with SATA and NIC support)

Later i create a new VM with

8 vCPU cores 1 socket

4GB RAM

60GB HDD

and the PCI devices (ATI audio, ATI R9 290 Hawai graphic card)

I installed Win8.1 Pro and installed the drivers step by step like the instruction from zyonov. But my R9 290 is not working code43 in the windows control panel and the monitor get no signal.

Any solutions?

Is it possible to stream the VM over the internal network to an other PC/Mac. The PC/Mac connected via VMWare Fusion, VMWare Workstation to the ESXi or alternativ with RDP RemoteFX or Splashtop.

I didnt want any monitor on the ESXi hypervisior. It will stand in the future in an other room.

[Docop]

Hi

So thanks Zyonov for info. I finally got the video working. But got big problem as to pass the USB. Which usb you pass, I got available the intel usb 1.0d and the 1.0a . I try to pass just the D and no result. Do both: and no result too. Did you set your main Bios in usb legacy mode too..?

On the GPU, here's a tips that may help people to not search too much for activating their GPU passthrough..

On my Asrock Z87m extrem4 , 16gig   - Bios as main display : PCIX and igpu disable. (onboard sound card and hdmi audio leaved at enable)

I setup my VM as -win7-64, 2vcor,2Gram,vmnic3, Boot bios as Bios

did the pci Hole addition line

boot the VM

install VMtools

restart

i put VMVGA in disable mode.

So i ran into a vgasave mode.

I first passthrought an Asus eax550 ATI card. All was detected without problem, driver install without problem too, all working, but never got video on the vm itself.

I try the intel onbard graphic, all perfectly detected, driver all installed fine. But never got any picture..

The esxi is booting fine, but after that, nothing more.

So i plug in an XFX 7750. I just deleted the driver on the vm. My xfx was detected fine in hardware manager. I now install the ati catalsy driver.

Reboot.

I log over windows esxi client, via console view. I got a big problem with the mouse as i can't control any VM with mouse integration. With or without vmtools. Mouse cursor just seem to disapear when i move it..

So i manage to use the keyboard to go over setting.

In resolution, i can't see other screen. Neither to log and go on catalys directly.   same thing wirh RDP

from the right click, on main desktop.

I go in CP / resolution display, click advanced, click the button catalys and now

i can got the catalys setup screen.

I manage to put same display on DVI and HDMI port.

And now all is working fine.

Other tips: You need an DVI monitor plug into the port. HDMI only will not work directly and hdmi connector is very flemsy, so use the dvi and after create many preset for each setup you want. All work without problem. But never with only hdmi...

For now, all the usb stuff.. i don't know yet. Hope that can help some for the gpu portion.

About rdp: i will tell what i've find later on. All audio and playing video is crappy over WM9. So I install DX11 and K-lite codec. I will test the result.

[rcriot18]

(UPDATED)

zyonov,

  Thank you for the info.

Was able to get ESXI installed on my hardware below and working. Even passed my GPU through to the virtual machine (Windows 7 x64)

1. Under the Advanced settings in ESXI. GPU showed up under the PCI pass through configuration page.

2. Passed Through the PCI for my GPU Radeon HD 7870 to the ESXI host(ESXI will make you pass both the GPU and the audio for the GPU which is fine).

3. Restarted the ESXI host.

1. Installed windows 7 x64bit on a virtual machine on ESXI.

2. Edit the virtual machine. Add a PCI device and selected my GPU that is being passed through to the host. Saved.

3. Booted up the virtual machine.

4. Installed the AMD driver for the GPU on the virtual machine with no problems.

5. Rebooted the virtual machine.

The GPU now shows up in device manager of the windows 7 virtual machine.

Issue:  Can't get the virtual machine display changed to output via the GPU without the virtual machine bluescreening.

Cause: Had the built in haswell gpu and multimonitor setting enabled in the BIOS.

Disabled them.

---Was still getting the blue screen on the virtual machine for some weird reason everytime I plugged in the monitor to the physical GPU.---

1. Booted up the windows 7 virtual machine with no monitor plugged into the physical GPU.

2. Went into device manager, Disabled the SVGA display adapter. (The console viewer in the VMware vsphere client went blank).

3. Plugged up the monitor to the physical GPU. The virtual machine finally popped up on the monitor that I plugged into the Physical GPU.

4. After rebooting the virtual machine with the monitor plugged in it booted up successfully with no crash. However, it was only displaying on the console viewer not the monitor and AMD Catalyst control center notified me saying no compatible device found.

5. Went back into device manager. The AMD GPU display adapter was still there. The SVGA display adapter was still showing disabled.

6. Re-enabled the SVGA display adapter in device manager and rebooted the virtual machine.

7. The virtual machine booted up into Windows just fine with the display plugged into the GPU. When it booted up it automatically outputted windows to the monitor plugged into the physical GPU.

(Thinking something funky with the SVGA driver helping the GPU passthrough output to the monitor. Not sure at all but it works now.)

After doing all of that it works now!!!

The windows 7 virtual machine displays on the monitor that is plugged into the physical GPU successfully everytime it boots up.

(There are two displays that show up in the windows screen resolution display manager: 1. The monitor that is plugged into the GPU and the display that is outputted to the esxi console viewer.)

I can go into the windows screen resolution display manager and remove the SVGA display output so it will only display on the monitor plugged into the GPU.

Issue now:  Can't get the USB controller to work in the windows virtual machine.

The pci for the USB controller (built into the motherboard) is configured to passthrough to the host. Added the PCI for the USB controller to the virtual machine. The enhanced host usb controller shows up in the device manager but it won't turn it on. The drivers for the USB 3.0 controller for my motherboard don't install as they say no compatible hardware found.

Thinking it might be better to get a PCI-e USB card and pass it through to the host for use with the virtual machine. Just not sure which ones will work with ESXI. Looking that up.

Hardware settings and configuration:

BIOS:

1. Enabled both VT-d and hardware virtualization. There are two different settings in the BIOS.

2. Disable Built in CPU graphics, multimonitor setting, and CPU audio output.

3. Make PCI-E as primary graphics.

Built in NIC on the Motherboard:

  Got ESXI to recognize the built in NIC on the motherboard just fine after following instructions GLRoman gave and used the .vib driver file he provided. Used the ESXI-Customizer to add that driver into the ESXI installer .iso. The built in NIC is working just fine for me now.

  1. Reference page for .vib file GLRoman provided: ESXi 5.5 ASUS H97M Plus, Intel I218-V "No Network Adapters" Error

  2. How GLRoman made a custom ESXI installer .iso with the .vib driver using ESXi-Customizer.cmd v2.7.2: ESXi 5.5 ASUS H97M Plus, Intel I218-V "No Network Adapters" Error

Specs:

CPU: Intel i7 4790K

Motherboard: Asus Maximus VII Hero

GPU: XFX Radeon HD 7870

Built in NIC on Motherboard: Intel I218-V (Working with a custom driver someone made for it to work with ESXI. Driver noted above)

ESXI Version: 5.5.0 (with custom driver for the nic card built into my motherboard. Noted above. )

[Docop]

Hi

I know it's painfull as ... terrible. But, be sure to not have any igpu or onboard graphic and no onboard audio.

I too got lot of problem,

Make many ova of you VM working and when it got your ati card detected.

VM should at leas be VM hardware 8.

You don't really need the pci.hole thing. Just put 5 gig of ram and be sure to click : reserve all memory.

After, deploy 3 time the same ova vm machine.

on each, install all different catalyst driver.

On my 7750, i got terrible time. The only driver working if the .9 version. Not the 12, omega or the new 13.5 beta. You may try older one too.

And as much as i don't like win8.1, it's works without problem vs win7 that cause always some hits. Also, don't do any update on your windows. Some update are proved to be causing issues on the Omega driver. The current stable one.

Hope that help. And at end, only working in HD, but Farcry work without any issues, Ryse son of Rome at full details run without any problem too. I seriously getting hard time to see what is the delay into the vm vs playing on the pc itself. Very fluid. But on car game with some smoke, like Grid, that cause some little lag or let say a 20fps vs 40 as normal.

And all is install and loaded from a usb drive, with a usb3 card startech passthrough. No delay and the only, most of time if just at startup like farcry

[rcriot18]

Hi Docop,

What is the exact usb3 card you have in your system w/passthrough to host?

I got everything working with the GPU in my virtual machine. (I updated my previous post). I just need to find a usb3 pci-e card to passthrough to the host for use with my windows 7 virtual machine.

[Docop]

Hi

So the card is a Startech pexusbs2, 2 usb port, pretty cheap and working without a problem, with usb hub it's best or with a cable extender it work too.

Be sure to put the power on the card too, if not, external hdd will not spin.

I might suggest to go with a usb 3.1, it might work, but don't know. it look the Asus 3.1 is about 5 to 10$ more that the usb3 Startech. If you want to give a try... If it's not working with esxi, you can got a full usb3.1 on your main computer. But it also only for win 8.1 or the new W10.

let us now later on of your results.

[rcriot18]

Finally got the built in USB controller to work in the windows 7 virtual machine.

The issue was the windows 7 x64 installed with the BIOS boot for some reason on ESXI 5.5.0 had issues turning on the controller when checking device manager.

Resolution:

  Re-Installed the windows 7 x64 with the boot option in the virtual machine set to EFI instead of BIOS.

After I installed it with the EFI boot option and installed vmware tools the USB controller worked great!.

Keyboard, mouse, camera, and wired usb 360 controller all popped up in the windows 7 virtual machine just great. No need to get the pci-e usb card.

Tested:

Played The Crew on the virtual machine with the same settings as I do on my native windows 7 and it played flawlessly with almost no drop in performance as well as no lag in the controls.

I got the audio over HDMI from my AMD GPU to work in the virtual machine as well. Will note later on that. (but simply just had to passthrough only the pci for the GPU Audio to the windows 7 vm and installed the GPU audio driver, and rebooted. It showed up in the device manager after the reboot. I then shut it down and added the PCI for the usb and the pci for the GPU. Now the GPU and GPU audio over HDMI to the HDTV work )

[phantazma]

I read this thread and decided to create a gaming VM. I just wanted to share my experience, finally got it (mostly) working after a week of messing around with it. For the first few days, I was getting all sorts of blue screens and constant sudden shutdowns but i think i finally got it.

Setup:

ESXi 5.5.2

Motherboard: MSI Z87-GD65

Processor: Intel i7-4790K 4.0Ghz

Windows 8.1

Sound: Sound Blaster Z

NIC: Intel EXPI9301CTBLK 10/100/1000Mbps

GPU: Radeon R9 280

My steps that finally made it work:

- In the motherboard BIOS, I ensured the the CPU had VT-D enabled. I couldn't "disable" the onboard graphics but I set the default graphics to PEG. I connected a monitor to the Radeon card via DVI cable.

- In vSphere, enabled passthrough devices (Configuration -> Advanced Settings -> Edit -> put a check mark on GPU w/HDMI audio, Intel USB EHCI, and Sound Card)

- Rebooted ESXi host

- Created Windows 8.1 VM. While still powered down:

* 4 processor cores

* 80 GB

* 5GB memory. Went to Resources tab and enabled "Reserve all guest memory"

* Kept Boot Options to BIOS (not EFI)

* Removed Floppy Drive

* Added all PCI devices except HDMI audio (enabled Radeon R9 280, Intel USB, Sound card)

* Set DVD to Windows 8 ISO and "Connect at power on"

- After all the settings configured, I went to the datastore and downloaded the *.vmx file to my local PC.

- Added the following to the end of the .vmx file and making sure there is no trailing empty lines:

pciHole.start = "1200"

pciHole.end = "2200"

- Uploaded the file back to the datastore

- Started up the VM and installed Windows 8.1

- Disabled Windows Automatic Updates.

- Enable Remote Desktop

- Windows automatically installed drivers for Radeon and Sound Blaster. Note that both display drivers are enabled. Installed VMtools but did a custom installation, removed the check mark for VGA drivers. Don't reboot.

- Downloaded and installed latest drivers for Radeon and Sound Blaster.

- Rebooted VM and hit F8 to go into safe mode.

- In safe mode, disabled the VMware display driver, leaving only Radeon enabled.

- Rebooted VM and closed the console.

I was able to remote into the VM and then logged out. I went to the server, connected a keyboard and mouse and I was able to use it like a normal PC. Verified that the VM was using Radeon by going to Start -> Run -> dxdiag. Went to the Screen Resolution, set it to 1920x1080, and disabled the other screens. Played League of Legends at highest settings and was getting 250-400 FPS. Installed Windows Updates.

** Problem: I set the sound to stereo only, disabled EQ and all extra features. Sound is good most of the time but it gets choppy randomly.

[MrWhippy]

phantazma‌

- Downloaded and installed latest drivers for Radeon and Sound Blaster.

Can you please let me know the specific AMD drivers you're using? What was the name of the .exe you downloaded?

Thanks!

[phantazma]

I downloaded from their official website, don't remember the name of the file:

AMD Radeon R9 200 Series

Driver Date: 11/20/2014

Driver Version: 14.501.1003.0

[kluc11]

here is my 5 cents on this, I invested a significant amount of time and effort in building esx based HTPC, here are my current specs (high level):

- esxi 6.0

- ASRock 970 extreme 4

- radeon 5450

- serveraid m5015

- 7x 1TB WD enterprise drives in RAID 5 config

- 850W power supply

- Windows 7 x64 VM - 8GB RAM, 4 virtual procs

- disabled vmware video adapter

- passed mouse and keyboard to the vm using synergy

- using latest catalyst version

Was able to pass through the video card to the VM and was able to see my desktop on the screen through HDMI output on the card (sounds over hdmi "works" as well)

My issue is that the sound and video are choppy to the point of not being usable, also, in WMC (windows media center) I'm getting an error stating that my video driver does not support protected content (HDCP). In WMC the video and sound playback starts and stops after 2-3 seconds with the error mentioned above.

I read some stuff about changing some acceleration settings in catalyst which I tried and didn't see any improvement.

Also, I still have to try reserving the memory for the VM and creating the pci hole in the vmx file.

I also ordered a new card: radeon R7 240 and a PCIe USB adapter and will try to pass those 2 through and see if I get better results.

Does anyone have any other suggestions specifically in regards to HDCP support in pass through?

How about using USB3.0 video card through a USB 3.0 card which is passed through?

thanks

[toysareforboys]

phantazma did you ever get the audio problem fixed??

-Jamie M.

[Luc_B]

All it fine for me.

ASUS-A99

I7 5930K

R9 380

R7 250

R5450

1: Activate in the Bios VT-d (Iommu)      (VT-d compatible CPU and motherboard require)

2: Installing ESXi 6U2

3: Edit VM with Win10

4: Passthrough video card(audio HDMI) and USB controler (asign to each VM)

5: install VMware Tools

VM Windows 10 detect the video card / audio / and USB (mouse / keyboard)

Yesterday I made a first test and I played at Wargame Red Dragon no lag next will be BF4

!

[Luc_RI]

Hello,

I've already managed to get VMs working with several PCIe Passthrough devices and was able to play games in parallel on one PC with two VMs and dedicated GPUs.

But what I realy want to do is a VM Windows10 with Fury X crossfire.

1.

Did anyone ever tried to get crossfire working in a VM and got it working?

Everytime, windows installs amd crimson driver, the display goes black and I have to shut down the host, to get things back working.

I'm also not able to install any crimson 16.x driver version when using only one fury x GPU. The system hangs.

Currently, only the Crimson 15.12 driver is working with ONE passthrough GPU.

I'm wondering, if the PLX PCIe Bridge chips on the Motherboard could cause the trouble?

Besides the VM crossfire issue, I'm also not able to install any crimson 16.x driver in a Windows 10 baremetal installation.

2.

Another problem, I could not fix by my self yet, is to reboot a VM with passthrough GPU.

After VM restart, the GPU is in an undefined state and doesn't work anymore.

I already read about ejecting any PCI devices before reboot, but I don't know how to do that.

I would be happy to get some input on that.

My setup:

ASUS X99-E WS/USB3.1 (newest BIOS with vertualization and VT-d enabled)

Xeon E5 1650 v3

2x 16gb Crucial DDR4-2133 Reg. ECC

2x AMD Fury X

LSI 9260-8i

M.2 SSD

4 SATA SSD

ESXi 6u1

Thanks in advance

[ce_Horizon]

I'm currently testing esxi with Windows 7 on my Haswell setup - thanks to all involved here, everything is now working well after hours of testing and BSODs over the course of several months.

Eventually I want to get two VMs running on their own video cards for 2 person use/gaming after I buy the required full ATX mainboard with 2x PCI-Ex x16 slots.

Hardware:

• MSI H87M-E33 mATX board with latest BIOS
• i5-4440 4core @ 3.1GHz
• 8GB Crucial Ballistix DDR3
• Sapphire Radeon R7 250 with boost
• WD Raptor 150GB spinning rust
• Corsair Nova 2 60GB SSD with "fixed" firmware
• Lite-on DVD/RW
• Lian Li "blue ring" full tower case
• Cherry USB keyboard and DELL USB mouse

Software:

• ESXI 6.0 update 2 on the Raptor
• Windows 7 x64 (vm version 😎 on the Nova 2 with all Windows Updates
• Radeon Software Crimson Edition 16.3.2