VMware Cloud Community
HellMind
Contributor
Contributor

Esxi 5.1 pci passthrough broken

I got a purple screen when I start up a vm with a pci device

http://ft.trillian.im/940e0c6710303f7966cf57a2bcc043251745ed62/6aBIiJHevbPWvWRlZ3Q8XDKJCNbhS.jpg

With Esxi5 was working fine.

I tested it with 2 vm on different hosts ( but same hardware)

http://ft.trillian.im/940e0c6710303f7966cf57a2bcc043251745ed62/6aBIHwqu0xJF5VWcS7HooQWupa9Ly.jpg

http://ft.trillian.im/940e0c6710303f7966cf57a2bcc043251745ed62/6aBIODLiCWadA8FYb57ZlMvwVctSh.jpg

http://ft.trillian.im/940e0c6710303f7966cf57a2bcc043251745ed62/6aBIWlBZlQXgImSaCnmpX2necw5yb.jpg

i7 2600 32gb

mb: dq67sw

What can I try

275 Replies
stevedparry
Contributor
Contributor

Thanks for the reply guys, even though I was hoping for a different answer. I'm assuming 5.0U1 is the last of the working versions?

Reply
0 Kudos
derickso
Contributor
Contributor

Im using all of the latest patches for 5.0 and it works fine.

Reply
0 Kudos
srwsol
Hot Shot
Hot Shot

Hi Mario:

Has VMWare given you any indication of whether they are treating this as a bug, or if they deliberately removed the ability?  I thought I read in a post somwhere that they told someone that the ability to pass through a USB device had been deliberately removed, and if the person wanted it back they would have to request it as a "new feature".  I sure hope that isn't true.

Reply
0 Kudos
kozzy3032011101
Contributor
Contributor

ESXi 5.0 up2 works fine for me.

I just did some testing with ESXi 5.1 build 1021289, I found the same issue as the previous posts.

Basic steps, tick the device for direct access, reboot, tick the device again after every reboot.

Hopefully in the next update they get it right.

Reply
0 Kudos
Mario1976
Contributor
Contributor

Hi srwsol,

the last thing was that I uploaded different logfiles (ESXi 5.0-914586, ESXi 5.1-914609 and ESXi 5.1-1021289) and complete guidance for reproduction for the third time to simply prove, that USB Controller Passthrough worked under ESXi 5.0 and doesn't work with ESXi 5.1 on the same hardware (HP DL360 G7).

According to the logs of the ESXi 5.1-914609 the BIOS seems to tell ESXi that Interrupt Remapping is disabled, and that was why the USB Hostcontroller wasn't put into Passthrough Mode during reboot. But there's no separate option in this BIOS and VT-d was enabled all the time.

Unfortunately since then only silence, no responses to the uploaded logfiles or following questions about the status... Wait since approx. 1 week after the last patch release now...

Reply
0 Kudos
RickFlowers
Contributor
Contributor

This is so bad for VMWare.

I have been trying for 2 months to get this resolved and my only option now is to completely refresh and rollback to 5.0 instead of staying on 5.1.

Thanks VMWare for breaking something as useful as USB 3.0! /sarcasm

Reply
0 Kudos
yeehawjared
Contributor
Contributor

I just applied ESXi510-201303001 and the issue remains.

Now the host takes a long time to start up, temporarily hanging on starting the usbarbitrator service.


http://hostilecoding.blogspot.com/2013/02/esxi-51-stuck-on-running-usbarbitrator.html

The host does come up eventually, but the DirectPath I/O Configuration still says I need to reboot to take effect.

Smiley Sad

VMware should really try to fix this - it's been about a year now.

Reply
0 Kudos
Bembel
Enthusiast
Enthusiast

Vielen Dank für Ihre E-mail.

Bis einschließlich Mittwoch, den 24. April, bin ich nicht im Büro und nur sporadisch per E-mail oder Telefon erreichbar.

Ab dem 25. April bin ich wieder im Büro erreichbar.

Mit freundlichen Grüßen,

Martin Sitte

Thank you for your e-mail.

I will not be in the office until April 24th and will not be responding to e-mails in time.

On April 25th I will be back in the office again.

Best regards,

Martin Sitte

Reply
0 Kudos
yeehawjared
Contributor
Contributor

Update:

it looks like ESXi510-201303001 may have solved it.  Although I get the reboot warning, I can add a PCI passthrough device on my VMs.

Reply
0 Kudos
Mario1976
Contributor
Contributor

Yes that is what was already stated above, that it "some kind of works" with this patch. But you'll have to reenable this USB-Hostcontroller for Passthrough after each reboot.

Not nice from a High-Availability Point of View...

There's still not much progress with our support case. They repeated their findings in the logfiles that they get the info from the BIOS, that Interrupt Remapping is not available, which is why the device is removed upon reboot, recommended updating BIOS or asking the Hardware-Vendor for a new BIOS. I'm on the latest available Firmwares for my Hardware, and since it's all working with ESXi 5.0, I doubt that it's directly related to the Hardware.

Still got no explanation from them why it then works after downgrading back to ESXi 5.0 or why it started working partialy again after this latest ESXi 5.1 patch.

Reply
0 Kudos
Mario1976
Contributor
Contributor

Out of curiousity, I tried the new ESXi 5.1 U1 (build 1065491).

Sad to say, but with this build the "original" ESXi 5.1 behaviour seems to be restored, means with this build you mark a host controller for Passthrough and it directly vanishes after the required reboot.  :smileyangry:

I'll update our case with vmware.

Reply
0 Kudos
srwsol
Hot Shot
Hot Shot

Hi Mario:

Thanks for keeping up with this.  I'm very disappointed at how VMWare has handled this issue, as it's not trivial.  It's almost as if they are looking for reasons not to fix this.  Regarding your earlier post about them saying that the BIOS interrupt remapping is at fault on your machine, I'd suggest asking them to try it on one of their own lab machines if they think the problem is with your computer BIOS rather than ESXi.  As far as I know no USB controller on any machine whatsoever can be setup as a pass through device under 5.1; which if you believe their explanation that it's a BIOS problem implies that every BIOS out there has a problem.   That explanation just doesn't hold water.  I'd also suggest giving them a link to this thread if you haven't done that already, so that they understand that this problem isn't some sort of isolated issue that only affects a few computers with some sort of odd configuration.

Unfortunately I only have the essentials package so I'd have to pay a couple hundred dollars for the "opportunity" to tell them that they have a bug.   It sounds like our problem here is convincing them that they actually have a bug.

Reply
0 Kudos
KWBS
Contributor
Contributor

same here.Smiley Sad Would be nice if you Update this thread, if there is any solution/suggestion from vmware...

Reply
0 Kudos
brianwwfong
Contributor
Contributor

my server also tried USB 3.0 card to do passthrough. it is also failed. I already give it up.:smileyangry:

Reply
0 Kudos
lawlerz
Contributor
Contributor

I updated to the latest 5.1 from 5 and I too had this issue. Reverting to ESXi510-201303001 at least allows the passthrough to work until reboot. Hopefully vmware provides a proper fix in the next update.

Reply
0 Kudos
AberDino365
Contributor
Contributor

Just to confirm the issue... I also updated to ESXi 5.1.0 update 1 and can no longer pass through the onboard USB controller on a Dell PowerEdge T320 (vanishes after host reboot). After reverting back to ESXi 5.1.0 build 1021289 at least it works again, as long as I remember to re-enable passthrough before I reboot the host Smiley Wink! Like many others I would also appreciate a proper fix for this...

Reply
0 Kudos
StarrateIT
Contributor
Contributor

HI AberDino, we are running an HP ML350 Gen 8 and with ESXI 5.1 update 1 and can't get USB option to stay enabled after reboot.

How are you getting it to work, because when we tick the USB Bus's we want to enable I thought we had to reboot before it would work. Once server is back up after reboot they show as unticked.

Are they actually working even though they don't show ticked?

We would like to find a work around until Vmware catch a wake up and fix this important feature!

please let us know how you making it work?

Reply
0 Kudos
Mario1976
Contributor
Contributor

The only 5.1 build where it seems to work at least for one reboot is 1021289 (the last Patch before Update 1), Otherwise the only option is to go back to 5.0, there it's working with all builds.

Our call at vmware has been closed. They confirmed the problem and that for our case none of the available workarounds (normal USB device forwarding or using a Network-USB-Server) would work, since the USB-Dongle we need to use inside the VM for a banking software, doesn't work when using this workarounds.

Using USB-Servers is always our first approach, since there we're not bound to the host hardware and can use vmotion. Using USB device forwarding and USB Hostcontroller Passthrough is always our last resort. For us performance is not that important as for other users in this thread using it for USB3-Hostcontrollers for Backups...

There is now a KB-Article http://kb.vmware.com/kb/2044963 regarding this issue.

I'll say it this way, vmware-Engineering was kind of surprised, that it even worked in ESXi 5.0. They continue to investigate for a fix for this issue, and maybe they've already identified the cause, but it'll be not available before Q4/2013 at the earliest, and even that's not for sure if they encounter any problems during testing.

So currently there's no other real solution than staying with ESXi 5.0 if you need USB-Hostcontroller Passthrough.

Maybe it helps to push the priority for this fix if other members here with active support contracts could also open a case with vmware.

Reply
0 Kudos
KWBS
Contributor
Contributor

Can also confirm that VMware ESXi, 5.1.0, 1021289 is working. We reverted last week and we have no problems so far.

Reply
0 Kudos
AberDino365
Contributor
Contributor

Hi Mario1976,

Thank you very much for the update. I use the ESXi free edition on my server at home and unfortunately I don't have a support contract to log this issue. Main reason I'm using USB DirectPath passthrough is because I've got a virtualised MediaPortal TV Server with a number of USB tuners, and performance/reliability is a lot better in this mode rather than using individual USB device passthrough. As you say, at least it works for one reboot with ESXi 5.1 build 1021289. Hopefully the guys at VMware will make this a supported feature as it is definitely very useful for a number of reasons.

Reply
0 Kudos