Someone has to know why this happens? VMDK IO over NAS at times out performs VMNIC IO over NAS via VMNIC? We have created various numbers of Linux VMs (RHEL 5.4 based) and tested VMDq (NetQueu) loading. And under the right conditions, VMDK IO across NAS, does out perform VMNIC IO across a typical mount point across the VMNIC.
We have a ESX host with 1 10g NIC, that is it. All traffic routes through that 1 physical NIC, be it VMNIC mounts or VMKPORT routed VMDK IO to our NAS storage frames.
This seems odd, since with all NetQueues engaged (7 of 8, since 1 is dedicated to VMKPORT). We find that the maximum throughput per NetQueue when full subscribed is about 1.1Gb/s. But for VMDK IO to out perform VMNIC IO, when both route to the same NAS infrastruture, the same storage frame in fact, VMKPORT must be prioritized, or actually has more bandwidth then other NetQueue?
Figure someone has seen this before? Of course we are generating a lot of IO, to across the VMs and the NetQueues when testing. If VMKPORT is just one NetQueue, then with full loading across all NetQueues, VMDK IO should be much slower all the time, given it is in contention with the other NetQueues engaged.
