USB Device Cannot Start (Code 10) on certain hosts but works fine on others

[ccfulton]

I've tried this on 3 different hosts and the device and driver works fine on 1 and fails to start on 2.

Host: Windows 10 19041 on all 3

Client: Windows 7 Ultimate 32bit

VMWare Player 15.5.6 in all cases

Running the same copy the client across all 3

Latest VMWare tools

The one host that works is a Lenovo x220 with a core i5 2520 processor.  The other two are newer PCs a Lenovo T460 i5-core6300 and a NUC desktop i5 7250.

Might also be worth noting that the x220 host is Windows 10 upgraded from Win7 and the other two were fresh installs of Win10

I mention the processor type because when starting up the VM for the first time on a new host, that is the only thing that the client detects as new, otherwise there is nothing that happens when starting the VM for the first time.

Using USB 2.0 in the VM, but have tried 1.1 and 3.1 with the same behavior.

I have tried older versions of Player with no difference.  Tried disabling Hypervisor & Hyper-V, no difference.  Tried running the device in a WinXP VM also, with the same outcome.

For what it's worth, this USB device is an automotive tool to communicate with a particular make of car.  I have other similar ones for different manufacturers running in the same base VM that work just fine on all of these same hosts.

I have a feeling there is some driver security / signing thing that the newer PCs are expecting that the old one is not checking for but I don't have clear evidence of that.  The drivers are signed, SHA1 as they are pretty old.  The signature expired in 2010 and the client is running with a fade date of 2013 (again same on all 3 hosts).

Really scratching my head on this one.  Any ideas appreciated.

[ccfulton]

Little more information in looking through the vmware.log for the working and non-working host.  They are almost identical excepting for a few things like the deviceID in the ARRIVAL and CREATED events.

The one notable difference is the last line in the non-working one:

2020-08-07T12:47:27.721-07:00| vmx| W003: USBG warning: can't configure device usbioerr 'INVALID_PARAMETER'

[ccfulton]

I set usb.analyzer.enable = "TRUE"

Started up the VM and plugged in the USB dongle.  Most of the beginning parts are identical between the two cases up to the point below.

Anyone know what the status=3 3 means in the USBIO message?

Have also been exploring usb.quirks although I can't find any real documentation about what the options are, just some "try this" "try that" examples.  But say I was playing with usb.quirks.deviceX does the below log mean I should be using device3?

Failing:

2020-08-07T15:54:19.738-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T15:54:19.738-07:00| vmx| I005: USBIO:  000: 80 00 00 00 00 00 02 00                         ........       

2020-08-07T15:54:19.738-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T15:54:19.738-07:00| vmx| I005: USBIO:  000: 80 00 00 00 00 00 02 00                         ........       

2020-08-07T15:54:19.738-07:00| vmx| I005: USBIO:  000: 00 00                                           ..             

2020-08-07T15:54:19.769-07:00| vmx| I005: USBIO: SetConfiguration

2020-08-07T15:54:19.769-07:00| vmx| I005: (1)

2020-08-07T15:54:19.769-07:00| vmx| I005:

2020-08-07T15:54:19.769-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=0 0

2020-08-07T15:54:19.769-07:00| vmx| I005: USBIO:  000: 00 09 01 00 00 00 00 00                         ........       

2020-08-07T15:54:19.769-07:00| vmx| W003: DUtil: skipping duplicate endpoint address: 1

2020-08-07T15:54:19.770-07:00| vmx| W003: USBG warning: can't configure device usbioerr 'INVALID_PARAMETER'

2020-08-07T15:54:19.770-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=3 3

2020-08-07T15:54:19.770-07:00| vmx| I005: USBIO:  000: 00 09 01 00 00 00 00 00                         ........       

2020-08-07T15:54:19.800-07:00| vmx| I005: USBIO: SetConfiguration

2020-08-07T15:54:19.800-07:00| vmx| I005: (0)

2020-08-07T15:54:19.800-07:00| vmx| I005:

2020-08-07T15:54:19.800-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=0 0

2020-08-07T15:54:19.800-07:00| vmx| I005: USBIO:  000: 00 09 00 00 00 00 00 00                         ........       

2020-08-07T15:54:19.801-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=0 0

2020-08-07T15:54:19.801-07:00| vmx| I005: USBIO:  000: 00 09 00 00 00 00 00 00                         ........       

2020-08-07T15:54:21.146-07:00| vmx| I005: USBIO: Up dev=2 'usb:0' endpt=81 stream=0 datalen=8 numPackets=0 status=0 0

2020-08-07T15:54:21.146-07:00| vmx| I005: USBIO:  000: 00 00 75 2c f2 7e 00 00                         ..u,.~..       

There is no further occurance of dev=3 in the failing case.

Working:

2020-08-07T16:02:26.371-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T16:02:26.371-07:00| vmx| I005: USBIO:  000: 80 00 00 00 00 00 02 00                         ........       

2020-08-07T16:02:26.371-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T16:02:26.371-07:00| vmx| I005: USBIO:  000: 80 00 00 00 00 00 02 00                         ........       

2020-08-07T16:02:26.371-07:00| vmx| I005: USBIO:  000: 00 00                                           ..             

2020-08-07T16:02:26.403-07:00| vmx| I005: USBIO: SetConfiguration

2020-08-07T16:02:26.403-07:00| vmx| I005: (1)

2020-08-07T16:02:26.403-07:00| vmx| I005:

2020-08-07T16:02:26.403-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=0 0

2020-08-07T16:02:26.403-07:00| vmx| I005: USBIO:  000: 00 09 01 00 00 00 00 00                         ........       

2020-08-07T16:02:26.403-07:00| vmx| W003: DUtil: skipping duplicate endpoint address: 1

2020-08-07T16:02:26.403-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=0 numPackets=0 status=0 0

2020-08-07T16:02:26.403-07:00| vmx| I005: USBIO:  000: 00 09 01 00 00 00 00 00                         ........       

2020-08-07T16:02:26.715-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=3 stream=0 datalen=1 numPackets=0 status=0 0

2020-08-07T16:02:26.715-07:00| vmx| I005: USBIO:  000: 52                                              R              

2020-08-07T16:02:26.715-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=3 stream=0 datalen=1 numPackets=0 status=0 0

2020-08-07T16:02:26.778-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=1 stream=0 datalen=12 numPackets=0 status=0 0

2020-08-07T16:02:26.778-07:00| vmx| I005: USBIO:  000: 01 00 00 00 1c 00 20 00 04 00 00 00             ...... .....   

2020-08-07T16:02:26.778-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=1 stream=0 datalen=12 numPackets=0 status=0 0

2020-08-07T16:02:26.809-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=82 stream=0 datalen=64 numPackets=0 status=0 0

2020-08-07T16:02:26.809-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=82 stream=0 datalen=4 numPackets=0 status=0 0

2020-08-07T16:02:26.809-07:00| vmx| I005: USBIO:  000: b4 01 20 00                                     .. .           

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO: GetDescriptor

2020-08-07T16:02:26.840-07:00| vmx| I005: (string, 3

2020-08-07T16:02:26.840-07:00| vmx| I005: , langId=0x0409)

2020-08-07T16:02:26.840-07:00| vmx| I005:

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO: Down dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO:  000: 80 06 03 03 09 04 02 00                         ........       

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO: Up dev=3 'usb:2' endpt=0 stream=0 datalen=2 numPackets=0 status=0 0

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO:  000: 80 06 03 03 09 04 02 00                         ........       

2020-08-07T16:02:26.840-07:00| vmx| I005: USBIO:  000: 22 03                                           ".             

2020-08-07T16:02:26.871-07:00| vmx| I005: USBIO: GetDescriptor

2020-08-07T16:02:26.871-07:00| vmx| I005: (string, 3

2020-08-07T16:02:26.871-07:00| vmx| I005: , langId=0x0409)

2020-08-07T16:02:26.871-07:00| vmx| I005:

dev=3 continues to communicate from this point on with different datalen and content.

[ccfulton]

Well, I was able to round up a 64bit driver for this dongle and tried it natively in windows 10 on both hosts.

X220 loads fine, the T460 gives a Code 10 natively just as in the Win7 VM.

The difference I can see is that the x220 has an Intel 6 Series C200 USB Host driver from Intel (native USB 2.0) whereas the newer machine us running the Intel USB 3.0 eXtensible Host driver from Microsoft.

So it is looking like this is not really a VMWare issue and it is down to something about the USB host differences.

Maybe someone on here can answer this one for me:

Does the VMWare USB Passthrough bypass the host driver entirely and talk directly to hardware?  Or does it still rely on the host driver?

Trying to confirm if this is for sure a hardware compatibility issue with the host machine or if it could be a bug in the MS driver.  Does the fact that it does not work in either the VM or in the native OS mean it's a hardware level problem?

[ccfulton · Accepted solution]

After some more digging I am pretty confident the problem is actually a bug in the Microsoft xHCI driver.

I was following sporadic reports of certain USB2 devices having quirks when used on a USB3 host controller, mostly from the early days of USB3, and the claims were that disabling USB3 worked in those cases.  Most recent BIOS doesn't have the feature to do this any more but turns out there is a register you can set in the Intel PCH that controls a hardware MUX to accomplish this called XUSB2PR.  When set it can force all the traffic through the EHCI host rather than default xHCI.

I found a Windows build of the PCI Utilities that are common in Linux and while able to read the register value it does not seem to be able to write the register even from and administrator level power shell.  I'm guessing this is something related to user space not having privileged access to the hardware at that level but I didn't try that hard to figure it out.  It didn't provide any error messages though, just silently failed and you can only tell that it didn't work by reading the register again.

Next attempt was to try running a Linux host on this very same PC since I expected Linux might allow more flexibility in using the PCIUtils to test out these ideas.  Installed Ubuntu 20.04 on a USB stick, added VMWARE Player, fired up the exact same Windows VM I had been using under Windows 10 and... it worked perfectly.  No need to force USB2 or any other funny business.

To double confirm, I set up a dual boot for the same Lenovo PC and did some more testing.  Seems at least as stable with the VM on a Linux host as the old x200 was running a Windows host and on the new laptop the Win7 VM even seems to be a little quicker running under Ubuntu that it is in Win10.

I am going to call this one "solved".

Maybe not the most convenient solution to boot into Linux but better than having to keep an ancient laptop around forever.  The whole point of it being in a VM in the first place was so that I could migrate it from PC to PC as time goes on.  This is an occasional use tool for me and I wouldn't be surprised if Linux ends up being a more stable platform to do this kind of thing on anyway.

There were no replies, but I see a reasonable number of reads so, thanks for following along

[wila]

There were no replies, but I see a reasonable number of reads so, thanks for following along

We're all bots :smileygrin:

Thanks for writing out your detailed analysis, I'm sure it will help somebody who bumps into the same or similar issues later on.

--

Wil