Hello,
Recently some of our application owners are starting to ask for servers with 8 vCPU and 32+ GB of RAM. I am trying to establish a policy on what the max vCPU and RAM we will support and looking for some justification for those limits. Without that, app owners will just keep asking for more and more.
For quite a while I have always understood that there is more overhead to SMP than single vCPU VM's. vSphere 5 was supposed to make some substantial improvements to that, but you still have the issue of the scheduler that has to concurrently find an open slice on a bunch of cores for these SMP boxes to execute their instructions. Obviously the more VM's and the more SMP VM's the harder it is going to be for the scheduler to find slices of the physical cores available and you will start getting increased latency. There are many blogs that address this in detail so I don't think I need to spend much more time on that.
We have two clusters right now. Our Test/Dev cluster is four hosts, two socket 4 cores (8 cores total) and 40 GB of RAM per host. Our production cluster is six hosts, two socket 6 cores (12 cores total) and 144 GB of RAM per host.
My first thought is that vCPU should be less than the number of cores in one socket. Otherwise, I assume, each SMP instruction would need to utilitze a full socket just for that one VM. I recall reading something about there being issues if a VM has more vCPU than cores because of multi-threading across physical sockets.
So I would be suggesting, max 2 vCPU in our test/dev cluster and 4 vCPU in our production cluster. In addition, I would think that there would need to be some justification for even 4 vCPU. Keep in mind these are generic rules and there will always be exceptions. For instance if you have a main line of business application that you want to virtualize and you plan to basically dedicate a host to it. My guidelines though are for your average application owner that some department or business units needs some new application or an upgrade, etc.
On the RAM side, I have not found any guidelines yet, My biggest thing is that if you allocate too much RAM to one VM, DRS is going to only allow a handful of VM's on that host and that will cause more cpu pressure on the other hosts. Now if the application actually needs that RAM and you give it too little then you are also going to cause unnecessary swapping. My initial gut reaction is something like max RAM should be no larger than 1/2 the host's RAM or maybe 1/3, but there is no data to support this, it is just kind of a gut feel.
So, I am looking for opinions. Do you have policies around max vCPU and/or RAM? Does anyone see any drawbacks or problems with setting limits?
Thank you,
Kevin
