ESXi Network Performance Tuning

[wings7351]

I have seen discussions similar to this but completely answered the question. I have ESXi 3.5 U3 installed on a Dell 670, two Xeon 3.4GHz CPU, 4GB memory, 1 on board Gigabit LAN port, one Intel PCI-X dual Gigabit LAN Port card installed as well. I have VM Management netowork using a physical port and the guest system (in my case a test Windows 2003 standard) to use a different phyiscal LAN port. Windows 2003 guest is configured to use Enhanced vmxnet with VMware tools installed. Within the guest network, there is a guest Vista, network adapter is e1000 with vmware tools installed. Based on a suggestion from another forum, I turned the HT off from ESXi configuration, unsure if this is really going to make a difference.

If I transfer files from the the Windows 2003 share to the vista, which is within the virutal switch and network, the transfer rate is 13.5MB/s, which is not great; if I transfer the same file to another physical machine, e.g., another client on the physicial network, the rate is 5MB/s (of course all connections are Gigabit.

What would be the potential reason cuased the network speed to be so much different between the virtual network and physical network? I would expect there are some difference but would not expect to be so much.

I thought about setting up a OpenSolaris ZFS to handle the storage seperately, but unsure how much performance gain I can get.

Any suggestions is appreciated.

[wings7351]

this actually leads me to think the problem is at hypervisor level instead of guest OS level. Is there anything I tweak on the vswitch?

[Dave_Mishchenko]

Have you checked the duplex / speed setting for the NIC both in the VI client and at the switch? Network traffic between VMs on the same switch don't go out to the physical NIC port and can run as fast as the VMs can handle the IO.

[wings7351]

Dave, last time I checked the NIC setting for the vswitches are all auto-negotiate. I have always assumed it is full duplex by default anyways, is it not? From the VI client it shows the vswitch physical NICs are 1000M. I will verify it again tonight.

Thanks.

[wings7351]

Dave, verified just now. It shows "1000 Full" I assume it is gigabit full duplex? I also checked the NIC it reads actual speed "1000MB Full Duplex". Is there anything else I should check? I basically left the vswitch configuration to the default settings.

Thanks.

[wings7351]

My vswitch shows the driver for the NIC is e1000, my VMs had Enhanced vmxnet and Flexible, is this a problem?

[wings7351]

today I tested the Vista and 2003 within the same vswitch, removed the phyiscal NIC from the vswitch and the transfer rate is 13MB/s (102Mb/s) which is far below what other is experiencing, what I see here is that even the virtual NIC and vswitch are not configured correctly in my server and the between the vswitch to the physical NIC there are something else as well.

[wings7351]

the same piece of equipment used to run 50MB/s (400Mb/s) when it was running Vista 32bit (not VM).

[Dave_Mishchenko]

What sort of disk I/O do you get in a VM now?

[wings7351]

Dave, I used SiSoftware Sandra to test it, it was 35MB/s; when I use IOmeter it showed different speed with different block size; when I test the network speed using Sandra, it showed 26MB/s to another VM in the same vswitch; 3.5MB/s to another physical machine.

Even the 26MB/s is not reasonable, the 3.5MB/s is extremely not acceptable.

Would you think it has anything to do with the strip size, block size and socket size? just a thought.

[wings7351]

Ok, did some further testing. I attached a SATA drive to the on-board SATA port and set it up as Virtual Compatibility RDM. I then set this disk as one of the windows 2003 harddrive. BTW, my VMFS is on raid5 that has 3 1TB on a AAR-2610SA on a PCI-X slot running at 66MHz. Here is my test result:

1) Within Windows2003, Copy from RDM to the raid5, which means writing to the raid5, average 25MB/s;

2) Within windows 2003, Copy from raid5 to RDM, which means reading from the raid5, average 35MB/s;

3) Copy from Windows 2003 vm to Vista vm that is connected through the same vswitch, transfer speed 13MB/s;

4) Copy from windows 2003 to physical machine, transfer speed 5MB/s

So it seems clear again that the vswitch is not running the way it is expected to run. What did I do wrong?

[akramsh]

Dear All,

Excuse my English since I am NOT a native English speaker.

I have merely the same HW setup, and experiencing the very same low network transfer rate.

Things go a bit better though if I force the speed/duplex on both the ESXi physical NIC and physical switch I have.

As for the VM 2 VM transfer rate, I was able to get higher transfer rates using linux 2 linux.

so this suggests that windows might need some tweaking to get higher rates on a single TCP connection.

I am still monitoring to get the solution for the physical to VM network issue.

Thank you