andrewa42
Contributor
Contributor

Best CPU Allocation

Jump to solution

Greetings All,

I am installing a server with 3 VMs, 1 domain controller and 2 Remote Desktop servers (for thin clients).

Hardware: Dell R515, 2 x 3.1ghz 8-core Opteron, 32gb RDIMMs, 3 x 300gb SAS as RAID5

All OS are Server 2008 R2 x64. I am using vSphere Essentials and have installed ESXi 5.1 on the host.

My issue: I'm a bit hazy on exactly how ESXi allocates cpu resources...what I want is for the RD server(s) to get a large majority of the CPU and the domain controller to get only what it needs to function (it is not the only DC on the network).

I only really wanted one RD server, but when I configured the VM it didn't want to let me give most of the CPU to that vm, saying I can only do 8-way. So my real question then is, do I need to give each VM settings reflecting the physical assets I want it to utilize, or is this an invalid approach and ESXi will give however much CPU bandwidth is available?

I've read some of the ESXi resource management guide, but I'm still not clear on the relationship between physical and virtual CPU, and what the real limits of my Essentials licensing are in this regard. This server is going into a busy production environment, so any guidance would be greatly appreciated 🙂

Cheers,

Andrew

1 Solution

Accepted Solutions
srwsol
Hot Shot
Hot Shot

Perhaps I can answer some of the questions for you.  A vCPU is basically a core or a hyperthread core.  The trick with assigning vCPUs is this:  However many vCPUs you assign to a VM have to all be available before that VM can run and conversely when a VM is running (i.e. it's getting its timeslice) the number of vCPUs assigned to it are tied up and cannot be used by another VM, even if the the VM to which they are assigned is idle and/or not using them all.  Also, ESXi is smart enough to assign vCPUs to physical cores when it's time for a VM to run such that it won't use the associated hyperthread core unless it has to because of the performance hit, although there are some settings you can use to tinker with this behavior.  As a practical matter what this means is that if you have several VMs each using quite a few vCPUs you run the risk of performance problems because they can't all be getting their timeslices at once due to a shortage of physical cores to map to vCPUs.  This problem is somewhat ameliorated by hyperthread cores in the case of a VM with several vCPUs tying up physical cores even though it's not using them all because in that scenario another VM which gets the hyperthread core, and this is allowed to run, will run at full speed.   What this also means is that if you have a number of VMs using several vCPUs your choice of hardware is better if you get a computer with more but slower cores rather than fewer faster ones, even though the total CPU power of both computers is the same. 

There are also a few tuning tricks you can use to help favor your high priority over and above the resource allocations, especially if that favored VM is using several vCPUs, and provided you have a bit of extra CPU capacity.   You can use the CPU affinity settings to dedicate one or more cores to a particular VM.  For example I have an ESXi box with 2 AMD 6 core processors (no hyperthreading).  I have one workload on it which uses 4 vCPUs, several that use 2 vCPUs, and several that use just one vCPU.  The 4 vCPU workload (the most important one) is a SBS 2008 VM that has lots of processes running in it because it's both a domain server and exchange server and thus at certain times needs 4 vCPUs to run well.  To give it an edge I've prevented all the other VMs, via the CPU affinity settings, from using the last two physical cores.  Therefore the 4 vCPU VM has two of the four vCPUs that it needs to run available to it at all times and only has to compete with the rest of the VMs for two of the remaining ten cores.  This sort of things of course only works if you are willing to devote a portion of the resources of the box at all times to your high priority workload even if that means that they sit idle if the high priority workload is idle and the rest of the VMs need CPU. 

View solution in original post

0 Kudos
11 Replies
KingofVM
Contributor
Contributor

Hi Friends,

best solution for you query is go for hotswap in both CPU and RAM, as i saw you have low memory in hardware, this would be a good option for you and also configure the aggressive session of every VM so that it will help you to handle the high bandwidth production.

0 Kudos

Hi Andrew,

1) Since your using vSphere Essentials license you can only assign 8-way vCpu only check below link

Compare VMware vSphere Kits for Virtualization & Cloud Computing

2) vCpu vs PCPU

Avoiding the Virtual CPU Dilemma – Overprovisioning vCPU to pCPU ratios « Virtua...

Hope this help your queries

Please consider marking this answer "correct" or "helpful" if you found it useful.
andrewa42
Contributor
Contributor

Thank you for those links, they were both quite helpful. Now the question becomes a bit different...

Suppose, as an example, that I have a VM for which I have assigned (in Settings) a single socket and 2 cores. It is the only VM running on the host (which has two physical sockets each with 8 physical cores). Is this VM able to utilize all the CPU power available, or will it be limited to whatever can be provided by 2 of the cores (6.2ghz/4 threads), leaving 14 cores essentially idle?

The software running on my RD server isn't really that CPU intensive, but there will be 35-50 sessions active most of the time and I don't want to leave resources unused...

0 Kudos
andrewa42
Contributor
Contributor

I don't see a setting called "aggressive", just sliders for "reservation" and "limit". Am I looking in the wrong place?

0 Kudos
Suppose, as an example, that I have a VM for which I have assigned (in Settings) a single socket and 2 cores. It is the only VM running on the host (which has two physical sockets each with 8 physical cores). Is this VM able to utilize all the CPU power available, or will it be limited to whatever can be provided by 2 of the cores (6.2ghz/4 threads), leaving 14 cores essentially idle?

No, the VM will not be able to use all the CPU power available it will be limited to 2 cores.

Please consider marking this answer "correct" or "helpful" if you found it useful.
0 Kudos
bluefocus01
Contributor
Contributor

Hi Andrew,

Are you really sure your RD-Server will be able to use all allocated CPUs. In my oppinion it is quite meaningless to entitle 8 vCores and more for a machine which uses less the 10% of them in average. Due to the mechanism how the hypervisors cpu schedular works a machine with low utilization but many entitled vCores will run fairly unsmoothly and stutteringly and this will result in bad user experiance. Allthough your host has enouth free cores the CPU readyness time will rise dramaticaly.

We e.g. provide many business applications for end users via Citrix XenApp. There are often more then 50 Users per XenApp-Server. Each XA-Server only has entitled 2 vCores. Its more then enough in most cases. There are only bottlenecks if there is a user session with an amok running application which wastes 100% CPU-Ressources. But in this case it might be regardless of how many vCores I asign.

There are ways to push down the readyness for such oversized VMs. But the better way is allways to size a VM sensable to it's workload. (As small as possible and as big as necessary).

0 Kudos
andrewa42
Contributor
Contributor

I've never seen the app which will be running, but based on what I do know (it is a browser based EMR program for pediatrics) I would say no...especially if these Opteron chips are as quick as they appear to be. However, as long as I only allocate as many cores as I actually have, shouldn't things run well?

0 Kudos
bluefocus01
Contributor
Contributor

Yes and no. By default no VM get assigned dedicated cores on a host. Therefore a vm only "sends" cpu time requests and the schedular tries to find the entitled number of free cores on the host. After the work has done all the pCores get released. Allthough there are enough free cores left this "search for free cores" process takes longer the more cores the vm has entitled. Due to this mechanism all vms shuffles arround on all cpu cores on the host.

But if you have only one host and more then enough cores there is a possibility to minimize this cpu time-slice mechanism.

Disable the Hyperghreaded Core Sharing of your (high performance) VM. Once assigned this VM will nearly have exclusive rights for all vCores and the cpu readiness will drop dramaticaly.

But aware of the fact: If you set this to all VMs. The overprovisioning behavior becomes worse on this host.

pastedImage_0.png

srwsol
Hot Shot
Hot Shot

Perhaps I can answer some of the questions for you.  A vCPU is basically a core or a hyperthread core.  The trick with assigning vCPUs is this:  However many vCPUs you assign to a VM have to all be available before that VM can run and conversely when a VM is running (i.e. it's getting its timeslice) the number of vCPUs assigned to it are tied up and cannot be used by another VM, even if the the VM to which they are assigned is idle and/or not using them all.  Also, ESXi is smart enough to assign vCPUs to physical cores when it's time for a VM to run such that it won't use the associated hyperthread core unless it has to because of the performance hit, although there are some settings you can use to tinker with this behavior.  As a practical matter what this means is that if you have several VMs each using quite a few vCPUs you run the risk of performance problems because they can't all be getting their timeslices at once due to a shortage of physical cores to map to vCPUs.  This problem is somewhat ameliorated by hyperthread cores in the case of a VM with several vCPUs tying up physical cores even though it's not using them all because in that scenario another VM which gets the hyperthread core, and this is allowed to run, will run at full speed.   What this also means is that if you have a number of VMs using several vCPUs your choice of hardware is better if you get a computer with more but slower cores rather than fewer faster ones, even though the total CPU power of both computers is the same. 

There are also a few tuning tricks you can use to help favor your high priority over and above the resource allocations, especially if that favored VM is using several vCPUs, and provided you have a bit of extra CPU capacity.   You can use the CPU affinity settings to dedicate one or more cores to a particular VM.  For example I have an ESXi box with 2 AMD 6 core processors (no hyperthreading).  I have one workload on it which uses 4 vCPUs, several that use 2 vCPUs, and several that use just one vCPU.  The 4 vCPU workload (the most important one) is a SBS 2008 VM that has lots of processes running in it because it's both a domain server and exchange server and thus at certain times needs 4 vCPUs to run well.  To give it an edge I've prevented all the other VMs, via the CPU affinity settings, from using the last two physical cores.  Therefore the 4 vCPU VM has two of the four vCPUs that it needs to run available to it at all times and only has to compete with the rest of the VMs for two of the remaining ten cores.  This sort of things of course only works if you are willing to devote a portion of the resources of the box at all times to your high priority workload even if that means that they sit idle if the high priority workload is idle and the rest of the VMs need CPU. 

0 Kudos
andrewa42
Contributor
Contributor

So *now* I understand 🙂


Thank you very much for your insightful answers!

I'm thinking I need to go ahead and set up several RD server VMs, each with a pair of vCPUs, and just let the scheduler do its job. I have more CPUs than I need to actually do the (current) job.

0 Kudos
Techonium
Contributor
Contributor

"However many vCPUs you assign to a VM have to all be available before that VM can run and conversely when a VM is running (i.e. it's getting its timeslice) the number of vCPUs assigned to it are tied up and cannot be used by another VM, even if the the VM to which they are assigned is idle and/or not using them all. "


This seems to be a common misunderstanding that is not accurate according to The CPU Scheduler in VMware vSphere® 5.1 Performance Study Technical Whitepaper [link]:

"It is worth noting that an idle vCPU does not incur co-scheduling overhead even with strict co-scheduling. Since there is nothing to be executed in the vCPU, it is always considered making equal progress as the fastet sibling vCPU and does not co-stop the virtual machine. For example, when a single-threaded application runs in a 4-vCPU virtual machine, resulting in three idle vCPUs, there is no co-scheduling overhead and the virtual machine does not require four pCPUs to be available."


I suggest reading that document in it's entirety to properly understand how CPU scheduling works.


Shane

0 Kudos