Unplugging all USB devices seems to work just as well as killing the usbarbitrator process.
Tried if I could narrow it down to one specific USB device, and found that the situation is improved (but not quite fixed) by removing...
...an empty cable.
My probably last update on the issue:
The problem does not occur when starting a VM from the library, instead of opening the vmx directly.
Sometimes VMware Player will be "Not Responding" for a while, but eventually the Library shows up, and everything works fine from there.
I believe the underlying cause is the same (and USB-related), but since this form of it is only occassionally annoying, I can live with it.