Hello.

I want to use VMware Server to split up different services on a small server. Since two of it's main missions will be router/firewall and local file server I won't settle for just any network performance. General CPU load is low on my old server so I'm mostly focused on network speed. I'm having some issues I hope someone could help me with. Any pointers or suggestions as to what I could try is appreaciated.

The motherboard is an Asus M2NBP-VM CSM with onboard nVidia LAN (forcedeth), and I have an additional Intel Gigabit NIC (e1000) installed.
CPU is an Athlon X2 (dual core) at 2.1 GHz.
Host OS is CentOS 5 with PAE kernel (4 GB RAM).
VMware Server 1.0.4 is installed.
Pretty much everything is set to it's default.
Network performance has been measured mainly with iperf and FTP.
Each VM is assigned only one vCPU, and 1-3 vNICs, bridged to either real NIC, or a guest only vSwitch.

The other computers in my network have variying performance levels regarding network speed, but I want to achieve at least a steady 100-150 Mbps both in and out (and this sometimes has to pass from one VM to a routing VM before getting to the real network).

FreeBSD is my main choice of server OS, so it's too bad VMware Server isn't compatible with a FreeBSD host. But my big issue is currently FreeBSD guests. The network performance I get out of a FreeBSD 6.2 or 5.5 guests connected directly to the bridged interface is at the most 80 Mbps in either direction. With a Windows XP guest I get at the very least 100 Mbps with the default 8 kB TCP window in iperf, and more than 150 Mbps with 32 kB TCP window.

Measured from the host directly I reached speeds of over 250 Mbps. This is almost matched with a Linux guest (I tried both CentOS 4.5 and Ubuntu 7).

Internal speeds are horrible as well. Transfers between Linux/Linux or Linux/Windows guests could be considered acceptable, around 150-200 Mbps. But FreeBSD to FreeBSD is 35-70 Mbps.

Running iperf via the loopback interface on the host puts out over 2.5 Gbps. A Linux guest can reach 500 Mbps.

Things I have tried:

• Turning on and off different offloading capabilities for my NICs on the host with ethtool (however the TCP segmentation offloading, tso, cannot be changed with ethtool for my e1000, and no offloading options exists for my forcedeth NIC).
• Connecting the computers directly bypassing the physical switch.
• All three different types of vNICs (vlance, vmxnet, e1000).
• Three of the four types of virtual networks (bridged, host only, guest only - not NAT).
• Confirmed achievable network performance in the physical host environment.
• Different settings for TCP window size.

Things I have yet to try:

• Disabling the builtin NIC and only using the Intel e1000.
• ESX reference performance - I'm currently trying to get an evaluation ESX server up and running on the same hardware using a different hdd (and some NFS storage), to see what kind of throughput I will get.

As a side note I would like to point out that I have reached 800+ Mbps in VMware Server 1.0.4 between two guest VMs running CentOS 5 and Windows XP on my laptop (2.0 GHz Core 2 Duo). But the host to guest transfers seem limited to 100-150 Mbps, regardless of using bridged, host only or NAT networks.

Any help, pointers or additional things to try are greatly appreciated.

/Zxinn