Have you tried watching the %DRPRX in ESXTOP ? see if any packets are getting dropped? Is this a gigE adapter? could also check the physical port and see if there are any errors present there.

Here is a couple of docs that may help.

http://communities.vmware.com/docs/DOC-5240

http://communities.vmware.com/docs/DOC-5500