Test Host:
Gigabyte MD80-TM0, current firmware
2x Intel Xeon E5-2687W v4 CPU
768GB RAM, 2400MHz
SSD datastore
Base installation of ESXi 5.5/6.0, no tweaking/modifications
1 VM guest (test guest as per below)
TestVM:
Windows 2012R2 (Window Server 2016 had the same result)
8GB RAM
1-20vCPU for testing
Benchmark tools: CPU-Z, Cinebench
Any CPU intensive benchmark I run, the vCPU is showing 100% utilization, but the host is only reporting about 40% CPU usage. This is seen in esxtop (%USED vs %RUN), CPU Usage %, Demand MHz vs Usage in MHz.... It seems like there is some throttling going on somewhere, but I can't seem to pin it down:
Test Host:
TestVM:
%USED about 40% of %RUN
Here we can see the VM is requesting 5803MHz but only getting about 40% of that request. What would be causing this?
Meanwhile, I've run this same test on a very similar host::
Production Host:
Gigabyte MD80-TM0, current firmware. Bios using default power settings (energy efficient)
1x Intel Xeon E5-2620 v3 CPU
384GB RAM, 2400Mhz
HDD datastore
~ 40 running VMs
ESXi 5.5U3 + patch, Active Policy (power management): Balanced
On this host, I get almost 2x the per vCPU performance vs Test Host 1. Test Host 1 is brand new and has no other guests setup, so there is no resource contention there.
Production host:
W2012R2TEST1:
%USED close to 100% of %RUN
The only significant difference I can see is going from 1 CPU socket host vs 2 CPU socket host with 2x the RAM. The motherboard make & model are identical.
Things I've tried:
- Turning off any power saving features
- Set power management settings to High performance in the active policy
- Changing snoop settings to Cluster-on-Die
- ESXi 5.5U3+latest patch, ESXi 6U2
- 1vCPU, 20vCPU, always ~40% CPU/MHz usage