Old post, but since there were no responses...
I am using 8GB Cruzer Fits on a pair of intel servers in my home lab. I am running ESXi v5.1U1 currently and have no issues with my network. I can't see how this would be a USB drive issue. More likely it is BIOS related. Maybe a bug. Mine work fine, so hope that helps you eliminate that from your list. Sorry I can't pin it down any further.