CPU reservation vs total capacity

[gongi]

Hi,

I have a small Vsphere 6.7 environment, without Vcenter, for test purposes with a maximum capacity of 26,4 GHz. The CPU has 8 cores with a clock of 3,3 GHz.

The virtual machines need CPU reservation due to the application that runs on them, and the total reservation for the VM is 24 GHz.

The problem is that I can't power on all the VM's because the system returns me the message that "The available CPU resources in the parent resource pool are insufficient for the operation."

Why does the Vsphere return the error message also if the total reservation doesn't exceed the maximum capacity? I've also read the https://docs.vmware.com/en/VMware-vSphere/6.7/vsphere-esxi-vcenter-server-67-resource-management-gui... guide, but I still don't understand the reason why.

Any suggestion?

Daniele

[depping]

if you reserve all resources the system is going to run out and will give that message. It is basically saying that you reserved more capacity than you have unreserved capacity available. Keep in mind, ESXi itself also needs resources. Also, every VM (besides what you reserve) also requires extra resources to be able to power on. 

What kind of apps are these that all resources need to be reserved?

[gongi]

Thanks for the reply.

Sorry for the question but how can I reserve more capacity than I have if the virtual machines total reservation is behind the total availability? 

And, which is the Esxi needed resources? Is there a way to calculate it or there is a fixed percentage?

Thanks

[vbondzio]

You either have to reduce the reservation of the VM or reduce the number of vCPUs for the VM if you insist on a full reservation / need it for Latency Sensitivity = High.

The calculation for the ESXi system and vim reservations depend on the host size and other factors, to check how much you have available to reserve, you can check the capacity of the user pool (where all the VMs go) via:

vsish -e get /sched/groups/4/stats/capacity

Note that ESXi _reserves_ resources that it ultimately might not _utilize_, still, those reservations are in place to enable ESXi to run even under severe contention and it isn't supported to change the defaults. If you are using a host without vCenter and just running the application for testing purposes, you could probably get away with reducing the hostd reservation as that is set based on an approximation of the maximum number of VMs that run on the host at a certain level of churn. Note that it is probably unwise to reduce the default reservations if you expect your tight fitting VM to do any significant amount of IO.

[gongi]

Thanks for the quick reply.

I'm not using the Latency Sensitivity setting, but just reserving CPU for the application to run properly (it's a distributed proprietary application that needs to have CPU reserved to work with decent performances)

I've executed the command with these results:

group-capacity {
cpu-reserved:21000 MHz
cpu-unreserved:298 MHz
mem-reserved:793228 KB
mem-unreserved:125564384 KB

So, means that on my host I can only reserve at maximum 21298 Mhz that is more or less 80% of the total capacity (26400 Mhz)

It's correct to say that Esxi let you reserve a maximum the 80% of the total amount of Mhz?

[vbondzio]

It really depends on the host size, I'd say ESXi by itself (no NSX etc.) usually reserves around ~2.5 cores worth, that is a bigger hit to smaller hosts. A reservation isn't doing anything until there is contention, if that is the only VM on the host, not much will contend with it.

[gongi]

Thanks for the reply.

Based on all the info, seems that my way to do the capacity plan is a bit wrong because I use the total amount of Mhz to decide how to size an infrastructure.

I use the formula ((Host CPUs * Core numbers * CPU Ghz) * Overbooking factor) to calculate the maximum Mhz available, and then, based on the virtual machines total reservation needed I divide it by the host/blade capacity to find how many hosts I need. The result is rounded up.

Is this a good way or not?

Thanks

[vbondzio]

It really depends on whether you mean _reservable_ or _usable_ capacity. Reservable capacity is what you are doing minus what ESXi is reserving. Usable capacity is what you are doing plus max all core turbo boost cycles, plus SMT/HT capacity, minus the small but variable amount ESXi non-VM worlds are consuming. I tried to explain it in the 2nd part here: https://via.vmw.com/MCL1635