Monster VM only using half of host CPU resources

[kellino]

OK first the facts...

*  ESXi 5.5 U2

* Host server(s) have 2 sockets with 8 cores each.  That's 16 physical cores per server and 32 when you factor in hyper-threading.

* One VM has 16 vCPUs allocated w 64GB RAM

* vNUMA is enabled and coreinfo.exe shows the proper logical to physical layout.

Since this host primarily services only this one "monster" VM we kept the vCPU count at 16 as we did not want to overcommitt with hyperthreading given the single large VM scenario.

Given this configuration, the host commonly runs at half the CPU load relative to the VM (we are using VMware provided metrics, but the guest OS reports numbers largely in sync with VMware).

For example, the VM is at 90% for 20 minutes and CPU is only 40-50% on the host.  During this time we see all physical cores in ESX being utilized within 10% of each other.

Throwing aside hyper threading, we were expecting one single VM to be more efficient with 16 vCPUs given the physical layout, but why is the VM at 90% when the host is at 40-50%?

Seems either our understanding of hyper-threading is wrong (meaning we need to add more vCPUs) or there is something else at work that prevents the VM from using more of the hosts resources.

Any ideas would be appreciated.  We would like to unlock more of the host server CPU. Thanks!

[kellino]

Working on little sleep but let me try to explain this back to see if I'm on the right page

I do get the part where the 16 physical cores are presented to VMware has 32 logical cores via hyper threading.  Since this is a one VM to host (mostly) scenario we provisioned for the 16 physical cores rather than the 32 logical.

Below is another ESXTOP/Windows screen. I had ESXTOP at 5s updates but you can see CORE UTIL is 73 which aligns with 72% on VM (and these numbers were aligning as I watched updates).

PCPU is the 32 logical cores and its returning lower numbers because this metric sort of wants you to be using HT (which we really aren't in this one VM - one host scenario).

So is it fair/accurate to say that when HT is enabled in BIOS, the metrics returned for CPU and Mhz align with PCPU (divided by 32) rather than CORE (16)?  Thus the vSphere metrics leave the (false) impression that only half the host is being used which isn't really true.

While this makes sense I'm still scratching my head over why vROPS says more Mhz is consumed by the VM than the host is using (2nd graphic).

[RobertCooper]

We have made the same observation in our environment and have not identified a clear reason for it. I would be interested to see the results of setting the Hyper Threaded Core Sharing options on the VM to None.