VMware Cloud Community
cojaxx8
Enthusiast
Enthusiast
Jump to solution

ESXi 5.1 Passthrough

Hello,

I know there has been a lot of talk about USB Passthrough is broken in ESXi 5.1 but does anyone if it has actually been fixed yet?

Thanks

Peter

1 Solution

Accepted Solutions
cojaxx8
Enthusiast
Enthusiast
Jump to solution

I was actually able to squeeze a little bit more time out of the Maintenance Window. My results below.

I attempted to pass through an onboard PCI USB2.0 controller and it would not work. When the VM got started a message came up saying "Device xxx is not a passthrough device".

So to summarise

IBM x3100 M4 Machine Type: 2582E4M

VMware ESXi 5.1.0, U2 (1483097)

Onboard PCI USB 2.0 controller - FAILED - Message "Device xxx is not a passthrough device".

Dedicated PCI USB 3.0 controller - SUCCESS

In my opinion the main point to take away from this is there is greater chance of success passing through a Dedicated PCI USB device than an onboard one.

Once again thanks to vNEX for his contribution Smiley Happy

View solution in original post

14 Replies
King_Robert
Hot Shot
Hot Shot
Jump to solution

Configuring passthrough devices


To configure passthrough devices on an ESX/ESXi host:

  1. Select an ESX\ESXi host from the Inventory of VMware vSphere
    Client.

    Note: If you have a chipset with VT-d, when you
    click Advanced Settings in vSphere Client, you can select the devices that are
    dedicated to the VMDirectPath I/O.


  2. In the Configuration tab, click Advanced
    Settings. The passthrough Configuration page lists all
    available pass-through devices.

    Note: A green icon
    indicates that a device is enabled and active. An orange icon indicates that the
    state of the device has changed and the host must be rebooted before the device
    can be used.

  3. Click Edit.
  4. Select the devices and click OK.
  5. When the devices are selected, they are marked with an orange icon. Reboot
    for the change to take effect. After rebooting, the devices are marked with a
    green icon and are enabled.

    Note: The configuration
    changes are saved in the /etc/vmware/esx.conf
    file.

    The PCI slot number where the device was connected is 00:0b:0. It is recorded
    as:

    /device/000:11.0/owner =
    "passthru"


    Note: 11 is the decimal
    equivalent of the hexadecimal 0b.

To configure a PCI device on a virtual machine:

  1. From the Inventory in vSphere Client, right-click the virtual machine and
    click Edit Settings.
  2. Click the Hardware tab.
  3. Click Add
  4. Choose the PCI Device.
  5. Click Next.

    Note: When the device
    is assigned, the virtual machine must have a memory reservation for the full
    configured memory size.
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Thanks Robert. I am aware of the process for setting up pass through, but there has been a lot of discussion about it being broken in 5.1.

I was wondering if it had actually been fixed by VMware yet?

Reply
0 Kudos
vNEX
Expert
Expert
Jump to solution

Hi,

5.1.x release has two bugs which are right now just a history...

First was PSOD on 5.1 hosts configured with PCI pass-through

http://kb.vmware.com/kb/2039030

Second was related to USB controllers (PR979390):

http://kb.vmware.com/kb/2062310

Both fixes are included in 5.1 U2 release (build:1483097)

Message was edited by: vNEX

_________________________________________________________________________________________ If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, P.
cojaxx8
Enthusiast
Enthusiast
Jump to solution

ah great, so looks like U2 fixes the problem.

I'll test out in the next few days and report back.

Thanks!

Reply
0 Kudos
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Hello,

I just updated a server to ESXi 5.1.0 Update U2.

The issue with the PCI device being lost after restarting has gone away but now it get this error after trying to start a virtual machine once the PCI device has been added in.

Device 000:26.0 is not a passthrough device.

The PCI device I'm trying to passthrough is a USB 2.0 controller.

Any thoughts on this?

PS. the build I'm using is 1483097

Reply
0 Kudos
vNEX
Expert
Expert
Jump to solution

Hello,

if your host is installed on HP Proliant server have a look at this official article:

Document ID:  c03835455

Advisory: (Revision) VMware ESXi 5.1 or ESXi 5.5 - The Virtual Machine Fails to Start and the Error Message "Device (PCI Bus/Dev/Fun) Is Not a Passthrough Device" Will Be Displayed

HP Support document - HP Support Center

_________________________________________________________________________________________ If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, P.
Reply
0 Kudos
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Hello,

Yeah I did see that article. The server is actually a Cisco though so not 100% sure if it still applies...?

Thanks

Peter

Reply
0 Kudos
vNEX
Expert
Expert
Jump to solution

Hi cojaxx8,

it applies only to HP Proliant servers...device you want to passthrough its onboard controller or PCI card ... unfortunately baseboard devices

have usually much less success with VMDP I/O unless they are listed as available.

Can you please post more info about your UCS server? because only one supported model for VMDP I/O regarding VMware HCL is this one below:

Cisco UCS - E160DP M1 Intel Xeon E5-2400 Series

_________________________________________________________________________________________ If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, P.
Reply
0 Kudos
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Hello,

The server itself is a Cisco UCSC-C240-M3S2. The device I am trying to passthrough is a onboard USB2.0 controller. Perhaps that could be the problem...

On Tuesday I am planning on upgrading a IBM x3100 M4 server running ESXi 5.0.0 Update 2. Installed in the server is a D-Link PCI Express card that is currently working successfully in PCI Passthrough mode to one of the VMs.

I am unsure at this stage whether to upgrade to ESXi 5.0.0 Update 3 or to 5.1.0 Update 2. Considering the issues I'm having with 5.1.x Update x I'm tempted to just upgrade to ESXi 5.0.0 Update 3.

The only reason I am considering updating to 5.1.0 Update 2 is because another host in a different branch office is running that version so would like to keep it the same.

Guess what I can do is upgrade the host to 5.1.0 Update to, test the pci passthrough of the dlink USB3.0, if it fails roll back the host to 5.0.0 Update 3. (as long as I make sure not to upgrade the hardware of the VM's...)

Reply
0 Kudos
vNEX
Expert
Expert
Jump to solution

Hi Peter,

VMDirectPath I/O capability mainly depends on Intel VT-d technology and its support on server CPU/Chipset...

I wish I could tell you better news but unfortunately for some reason even IBM x3100 M4 isn't listed as compatible with VMware VMDirectPath I/O ...

See VMware HCL for details:

VMware Compatibility Guide: System Search

Can you verify CPU installed and chipset used in your IBM server?

_________________________________________________________________________________________ If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, P.
Reply
0 Kudos
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Hello,

The IBM Model is a 2582E4M, Intel Xeon E3-1270v2 and Intel C202 Chipset.


That compatibility guide is odd. It basically says no IBM Servers support VM Direct Path IO. I've got 10+ servers running a USB PCI device in passthrough mode and it works OK. I guess the compatibility guide only lists 'approved features'. The fact I have it working on so many servers is probably just luck.


I still might try updating it to 5.1.0 U2 on Tuesday and see how it goes. Since it's not officially supported I guess there is no way of knowing apart from actually doing it!

Reply
0 Kudos
vNEX
Expert
Expert
Jump to solution

Hello Peter,

Yes HCL outputs may appear odd in some cases but I think they aren't...in your case HCL just says that IBM as a vendor doesn't TEST  its servers for "VM Direct Path IO" this  is always a vendor initiative within TAP program.

At the end of this process each vendor hardware undergoes comprehensive testing by VMware hired 3rd-party company. (e.g. AppLabs acquired by CSC ... Certify Your Product with CSC's Testing Services )

So if an old or some new hardware isn't listed it doesn't mean it couldn't work with VMware products. It could be just an indication that its vendor doesn't apply to have its HW on VMware HCL.

(such piece of hardware doesn't gone through any extensive testing against VMware products or features)

... hence it  cannot be classified as certified (VMware Ready) and you cannot derive any support in relation to it from VMware GSS because its simply unsupported.

Apart from support ... because of a lack of complex testing you can encounter unpredictable behavior or crashes etc. ...  none of us want to see such hardware in production environment.

For Lab/Test environment its absolutely acceptable ... but not for production.

So yes as you mentioned in these cases its all about luck ... (especially in production also about braveness and responsibility of each admin 😉

In terms of updates I would always go for the latest maintenance releases.

Anyway your experience could be very valuable for others so if you will have success with IBM servers and VMDP I/O please ping back with results.

Thanks

_________________________________________________________________________________________ If you found this or any other answer helpful, please consider to award points. (use Correct or Helpful buttons) Regards, P.
cojaxx8
Enthusiast
Enthusiast
Jump to solution

Hello,

Well I have success!

The Host has been upgraded to ESXi 5.1.0 U2 1483097 and the PCI Passthrough is working successfully.

The PCI passthrough device is a dedicated PCI USB 3.0 controller, NOT an onboard device. I have not had time to test whether passing through a onboard device works as well due to a short Maintenance Window.

When I get the opportunity I will test for others benefits.

Thanks to vNEX for the very thought out responses.

Reply
0 Kudos
cojaxx8
Enthusiast
Enthusiast
Jump to solution

I was actually able to squeeze a little bit more time out of the Maintenance Window. My results below.

I attempted to pass through an onboard PCI USB2.0 controller and it would not work. When the VM got started a message came up saying "Device xxx is not a passthrough device".

So to summarise

IBM x3100 M4 Machine Type: 2582E4M

VMware ESXi 5.1.0, U2 (1483097)

Onboard PCI USB 2.0 controller - FAILED - Message "Device xxx is not a passthrough device".

Dedicated PCI USB 3.0 controller - SUCCESS

In my opinion the main point to take away from this is there is greater chance of success passing through a Dedicated PCI USB device than an onboard one.

Once again thanks to vNEX for his contribution Smiley Happy