There are 3 CPU metrics that you need to know about for CPU
CPU usage = The amount of CPU that the VM is using (getting from the host)
CPU demand = The amount of CPU the VM is demanding from the host
CPU Contention = The difference between the above 2.
CPU contention has nothing realy to do with the cpu usage of the cluster it points to 3 things in my experience
- To many VMs in the cluster
- To many vCPUs assigned to the VMs in the cluster
- Power policy on the hosts not set to high performance
The first two are due to VMs not being able to be scheduled the vCPUs on the physical cores because there are to many.
The second one is because the physical cores are powered off (and the cache depending on what it is set to) when not being used and the lag caused when a VM needs to use a powered off core can generate contention.
If your hosts power is set to high performance then you will need to reduce the number of vCPUs in the VMs or move vms off the cluster. I would only worry if you are experiencing performance problems.
The main reason i see contention getting out of control is someone complains that a VM is not performing so the first port of call is always "we need to double the cpu". The VM is blindly upgraded which generated more contention causing the VM to run slower. This is where right sizing is impotent. Only give a VM more vCPU if it needs it
- To many VMs in the cluster --This is strange because there is another clouster having more VMs but we still dont have the CPU contention problem. One host has only two running Virtual machine and has CPU contention of 4.5% which is strange.
- To many vCPUs assigned to the VMs in the cluster -- The Host has 80 vCPU and still only two Virtual machines are running in a host.
- Power policy on the hosts not set to high performance -- This is confirmed to be set to high Perfomance Mode.
Not sure why this is a case and what exactly can be the Impact of CPU contention in Virtual machines.