3 Replies Latest reply on Mar 26, 2015 2:58 PM by trunolimit

    ESXi 5.5 passthrough trouble

    zotto Lurker

      Hello,


      I bought Intel NUC 54250WYK device. I installed last ESXI 5.5 update1 and I would like have the following setup (I am beginner with ESXi):

       

      VM1: Windows 8.1 x64 with GPU passthrough

      VM2: XENology (NAS) with USB passthrough

      VM3: Ubuntu or other linux

       

      http://www.intel.com/content/www/us/en/nuc/nuc-kit-d54250wyk.html

      NUC has VT-d: enable

      NUC HDMI port is connected to Samsung TV 46C6000

       

      For passthrough mode I used the following guide: http://www.webbosworld.co.uk/blog/?p=471

       

      At first I installed VM1 machine without passthrough GPU = OK

      When I add GPU to passthrough mode, system is unstable. In first case VM1 boot to windows, and after 30 seconds it froze. I restarted VM1 to safe mode, there is no problem with boot. In device manager I could see two VGA device (VMware SVGA and Intel HD 5000 - see attached picture ESXI-04.png). I disabled WMware SVGA and restarted VM1. But in this setting, system didn’t start boot (only black screen in console and TV showed no signal).

       

      I google a couple of day and used the following advice without any success:

      • Setting pciHole.start = “1200″, pciHole.end = “2200″ and allocation of all memory to guest
      • I tried use EFI or BIOS for installation VM1

       

      Please can anyone help me to setup VM1 with GPU passthrough? Do you have any tips, which I should try it?

      Many thanks for any reply

        • 1. Re: ESXi 5.5 passthrough trouble
          Linjo Champion
          User Moderators

          I have never seen or heard of anyone that have successfully used the builtin Intel GPU with PCI Passthrough so the chances to get this to work is slim...

           

          // Linjo

          • 2. Re: ESXi 5.5 passthrough trouble
            merlinthemagic Lurker

            I have successfully built about 10 hosts with GPU pass through as of today. However it is so incredibly frustrating trying to gather all the little pieces of information needed to get up and running the first time, so here are my summarized notes.

             

            I suggest performing all the steps below, dont try to piecemeal it. Better do all the steps and then i.e. if you want to try and free up your integrated graphics card, do it after once you have a working setup.

             

             

            Hardware:

            1)     Motherboard AND Processor must support Intel Directed I/O (VT-D) or AMD IOMMU. If you are just trying to pass through a capture card or a storage controller i have found that it sometimes work even if the processor does not support VT-D, but for GPU passthrough both must have support.

             

            2)    The  video card for passthrough must be Radeon (AMD) and not NVidia. I have successfully passed through HD: 8500-7750, 6450, 5450 series cards. NVidia simply will not work, there are high-end dedicated cards which will work, but that is hardly relevant here.

             

            3)     You will need 2 video cards. One Radeon and the second can be a NVidia card (or another Radeon), since it will not be passed through but rather is used for ESXi console output.

             

            4)     USB controller. Save yourself a headache and buy a dedicated USB controller you can passthrough for keyboard and mouse. I have been using 'NEC uPD720200' (amazon $14) with a USB hub, that works great.

             

             

            Hardware Install:

            Do not place the video card you intend to passthrough in the primary graphics PCI-e slot (sometimes called PEG), this is usually a 16 lane PCI-E slot closest to the processor. This slot should be used for the ESXi console video card. Then place your passthrough card in one of the other slots. Some motherboards are pretty finicky and will only support passthrough on some but not all of the remaining slots, but that is not a concern right now, just pick one.

             

            BIOS:

             

            1)     Update to latest BIOS version. There are rare instances when an older BIOS works and newer do not, but it does not happen often.

             

            2)     Disable IGP (integrated graphics processor) and integrated audio devices. These on-board devices tend to interfere with passthrough, Especially IGP, so disable the IGP and tell the BIOS to use the new card you added for ESXi console access (most likely called PCI-E / PEG).

             

            3)     Enable Intel Directed I/O (VT-D) or AMD IOMMU. Sometimes called VT-D.

             

             

             

            Configure ESXi for passthrough:

            Boot into ESXi and use vSphere to access the Configuration -> Advanced and check off the Radeon graphics card, USB controller and any other device you wish to pass through. WARNING: do not pass through the controller (USB / SATA etc) that holds the ESXI install! If you do this will be the last time you can edit the settings of the host as the storage device with the ESXi config will become unavailable for the ESXi OS after boot, it is a real pain, so take care.

             

            For Passthrough to take effect reboot the ESXi host.

             

             

            Create VM:

            Do your normal routine when deploying a VM. Do not boot it.

             

             

            Configure VM Settings:

            Edit the VM configuration:

             

            Add the Radeon video card, if the video card has a HDMI Audio device add that to your VM as well, add the USB controller.

            Click OK.

             

            Edit the VM again. You should now see 2 or 3 PCi devices added, most likely called 0,1, 2 (note down the numbers). Then access the Options -> General -> Configuration Parameters menu and hit add row at the bottom.

             

            Add the following lines to the config:

            pciHole.Start = 1200

            pciHole.End = 2200

             

            Only add this config for the PCI device ids you noted down, in this case 0,1,2

            pciPassthru0.msiEnabled = FALSE

            pciPassthru1.msiEnabled = FALSE

            pciPassthru2.msiEnabled = FALSE

             

            Click OK.

             

            Edit the VM again.

             

            Change the RAM amount to 1 or 2GB. I struggled with this for a long time with a Win7 64bit VM i created starting with 8GB RAM, all devices would just error in device manager with "This device cannot start" Code 10. Lowered the RAM to 2GB and uninstalled the device from the device manager and it came up without issue. So start with 1-2GB RAM.

             

            Last access the Resources-> Memory and check the box for "Reserve All Guest Memory".

             

            Click OK.

             

             

            Boot the VM:

             

            Boot the VM. At this point the video card and usb controller should show up and install as in a non-VM environment, with a few caveats. Install all drivers, do this by downloading the newest drivers from AMDs site and installing without the "Catalyst Control Center" option, if you need the control center to remove the overscan "black frames", then install it separately after a reboot. Once you are done installing test your keyboard and mouse then shutdown the VM.

             

            Now change the RAM amount you would really like the VM to run with and boot into your new passthrough desktop.

             

             

            Performance:

            You will severely limit performance if msiEnabled remains set to FALSE for all devices, as we did above. I suggest trying to set this setting to TRUE for as many devices as possible. However certain devices do not work well when set to true, I found that HDMI audio device tend to sound horrible if msiEnabled=TRUE. However if the GPU is running with msiEnabled=FALSE you will have terrible performance with hardware accelerated flash videos. So TRUE for GPU devices and FALSE for audio .

             

             

             

            Done.

             

             

            Trouble Shooting:

            If you are installing multiple Video Cards for the same VM i have only had luck when using different models .i.e. a 7750 and a 6450.

             

            I gave one example above how to avoid the "This device cannot start" Code 10 error, however mostly this error is caused by the Radeon driver not installing correctly. Fix by completely removing the AMD packages (including Control center) and re installing.

            • 3. Re: ESXi 5.5 passthrough trouble
              trunolimit Novice

              I wonder If he ever got this to work.