Poor VM Bridged Performance compared to Virtualbox

[dsmithuk2]

Hi all,

I run pfsense in a VM, 3 comparative speed tests (all repeatable). I ran into numerous performance issues on vmw 10+11, and tried 11.1 today. The performance issues are still there, this is bridged mode from the WAN side, to the LAN side on W2008R2 on a HP N54L Microserver.

3 scenarios attached:

1) Virtual box (nearly 160mbit / 10mbit)

2) VMware Workstation 11.1 with e1000 (64 / 4)

3) VMware Workstation 11.1 with VMXNET3 (74 / 4)

I've tried some other tests which also seem to indicate the VMXNET3 vastly underperforms the Virtualbox one, usually by running out of CPU headroom (i.e the vmware bridged driver appears to be loading up the CPU's more than the VBox ones).

Anyone else seen this, and perhaps know anything to reduce CPU penalty on the VMWare driver?

thanks a lot. Dave.

[wila]

Hi,

That's a huge difference.

Is pfSense on VirtualBox configured exactly the same, and are you testing with exactly the same version?

I ask because pfSense was updated recently where the claims are that the firewall is faster.

Another reason I ask is because you mention high CPU. This is something I normally only see on pfSense when enabling promiscuous mode and when there's a lot of other traffic that the firewall then is looking at as well.

--

Wil

[dsmithuk2]

Hi,

thanks for the reply. I've tried fresh install on vmware and even an .ova export from vbox. Neither produces any higher performance.  Without a doubt the processor in the N54L is a bit weedy, but it doesn't explain the (vast) difference in performance between the two products. I've tried fresh uninstall/reinstall of vmware too.

Here's a pic showing the kind of loads experienced during that speed test... (the test is running from my workstation , whilst the performance stats are from a RDP session to the box).

thanks, Dave.

[juanjoelii]

Hi,
Are you sure this is all to blame to the CPU intensive use?
Maybe a different configuration on the networking side?
The workstation one runs alone or is sharing resources?

Anyway, it kinda bit me curious and im trying to set up something similar in my workstation. (i use Workstation 10)

Wich webserver do you use?

Regards.
Joel.

[dsmithuk2]

Hi,

there is similar spiking, but with VMware, but with much lower performance.

The crucial one (the WAN side) is same as VBOX, and only has the bridge protocol enabled, nothing else.

Regarding webserver - what are you speaking about here? the speed test site? if so it's speedtest.net

thx.

[wila]

Hi Dave,

dsmithuk2 wrote:

Here's a pic showing the kind of loads experienced during that speed test... (the test is running from my workstation , whilst the performance stats are from a RDP session to the box).

Unfortunately your screenshot doesn't show me any details as to what process is running high CPU. I take it that it is the pfSense process that is running the high CPU? The speedtest code is in flash and flash isn't exactly CPU friendly either, especially not when that part is run from within a VM.

For the record I just ran the test here (but I'm on Fusion 7.1.1, not workstation) and on my 60Mb/sec fiber I get a respectable 60Mbps down, 52 Mbps up.

When I then look at the CPU usage of the pfSense NAT firewall that is inbetween it initially goes up to 39% for a bit after which it goes down to 20% while still running at full speed. The NICs are "e1000" and memory of the VM is set to 272Mb, single CPU, pfSense 2.2-RELEASE (i386)

Not the same test obviously due to using different host OS+product, but same technology and just mentioning it as a reference.

I reran the speedtest on physical Windows 8.1 without pfSense inbetween the box and the internet and the numbers for that are 60 Mbps down, 59 Mbps up.

While I do see a small difference on the upload speed it might be due to other reasons as virtualisation (for ex. disk speed) without more research to isolate that it is difficult to say.

Your virtualbox vs. vmware test does sound interesting as a base reference though as it does appear to eliminate most other factors.

--

Wil

[dsmithuk2]

Hi Wil/All,

Forgot to loop back on this. Yes as mentioned the flash part (speed test) was done from a separate machine (my home pc) whilst the server which is running the VM is the one showing the high cpu load for the relevant VM Service (vmware-vmx for vmware).

Anyone got anything to add? I really want to move back to VMware workstation for the functionality, and also the stability, but obviously taking a 50% performance cut is not possible.

thanks a lot.

[dsmithuk2]

Problem now solved after switching to ESXi 5.5. Never got to the bottom of why the performance of VM workstation was so poor though.