ESXi Performance / Sizing Guide

[CTSTech]

Is there an easy guide somewhere that covers all of this?

• How to size vCPUs vs. pCPUs
• How to size Memory per VM
• How to check performance of VMs
• Which performance counters are important to check
• Performance counters - ranges of good, OK & bad
• When to add or remove vCPUs for better performance

We would like to learn how to tune ESXi for better performance - even if we have to change the configuration of VMs.

[CTSTech]

Anyone?

For example, how would you size Memory and vCPUs for this setup:

Hardware:

Server - HP DL380 G7

2x Xeon 3GHz CPUs - 6 cores each, plus Hyperthreaded

64 GB Ram

Local Storage - SAS RAID-5

VMs:

Server 2003, App Server, DHCP, DNS, Print Server, File Server, Citrix Server

Server 2008 R2, Domain Contoller, DNS, RDS Connection Manager

Server 2008 R2, File Server, WSUS

Server 2008 R2, Terminal Server 1 for about 15 users

Server 2008 R2, Terminal Server 2 for about 15 users

[joshodgers]

For sizing of VMs, I always use a capacity planning tool, such as VMware Capacity Planner.

The rule of thumb, is "Start small and scale up". (This is a concept of Right Sizing)

What this translates into is, every VM starts with 1vCPU unless there is a documented justification for additonal CPUs/RAM

A basic example of a justification would be an application that is multi threaded (such as SQL/Exchange/IIS) along with capacity planner or Windows perfmon stats showing an average of more than 1 cores worth of CPU usage.

Regarding RAM, the same applies, Start small and scale up if required.

To enable this, assuming a supported operating system such as Windows 2008 R2 x64, enable Hot Add and Hot Plug on the VM so you can add CPU/RAM without shutting down the VM. Note: You cannot hot remove CPU or RAM.

I would suggest using vCenter Operations for performance monitoring, as it tells you how everything is performing and where any bottlenecks are in a very easy to use GUI.

If you dont have vC OPS, you can use the vSphere client performance charts.

Generally, CPU Ready % is a common issue, so ensure that value per vCPU is well below 2000ms (which is ~ 2.5% CPU Ready).

If you have a CPU Ready issue, it basically means you have CPU scheduling contention, so adding more vCPUs will make the problem worse.

If your VMs are not using a large percentage of the resources they are assigned, downsize them, and you may be surprised that performance increases especially in environments with contention.

[Gkeerthy]

• How to size vCPUs vs. pCPUs

- this is purely based on your requiments and the capcity planner report of the existing physical servers, what kind of applications etc. IF the applications are multithreaded then giving multiple vCPUs give good performance. Like oracle, mssql, exchange etc. But in an esx host, the total vCPUs given should not be greater than total cores (physical cores + virtual cores) of the pCPU, ideally vmware recommends dont count the virtual cores, that is when you enable HT you will get more cores, when you deploy VM's you need to consider you have only physical cores. Example,if you have 20 physical cores, then 20 VMs 1vCPU per vm, you can little bit over commit the esx host by taking the virtual cores also.

Another thing when you give vCPU to vm are, if you give more vCPU to vm and the vm is not using it then you wil get poor performance because of co-scheduling. So only give multiple vCPUs if the vms can use those vCPUs to 80%

• How to size Memory per VM

it is good to have more pMemory in the esx host, and OS like 2008/2008 R2, and applications like exchange, mssql, it will take all the memory given and it will reserve it in the OS it self, so that the app can use it, and most of the operations will happen in the vm RAM and it will reduce the IOPS load in the storage, So it is good to allocate some extra memory to these vms

• How to check performance of VMs

vcenter and the esxtop will give the entire view of the infrastructure

refer the

http://www.yellow-bricks.com/esxtop/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=100820...

http://communities.vmware.com/docs/DOC-9279

http://www.petri.co.il/esxtop-overview.htm

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101792...

• Which performance counters are important to check

in the above link it is mentioned

http://www.vkernel.com/resources/topics

http://www.vkernel.com/vcpu-and-cpu-management-resources - "The Top 20 VMware Performance Metrics You Should Care About"

• Performance counters - ranges of good, OK & bad

refer above

• When to add or remove vCPUs for better performance

again, simply giving more vCPU wont deliver perfromance, co scheduling issue will come, for multithreaded application give 2 vCPUs and monitor through vcenter, veeam, VCOPS etc, then see the usage then give more vCPUs, the rule of thumb is if you give more vCPUS the vm must use to atleast 80% of all the vCPUs

For esx tuning, refer the resouce management guide, performance best practices from the VMware, you can download it

[CTSTech]

Thank you guys for the detailed answers.

For the Server scenario above... How much Memory and vCPUs would you start out with for the 2 Terminal Servers?

[joshodgers]

As you have 6 core CPUs, your optimal VM sizes are 1,2,3 and 6 as there can be divided into 6 with full numbers.

I would suggest your terminal server/s should be 2 vCPU and have additional terminal servers, rather than increasing vCPUs beyond 2 in your environment.

Regarding memory, that always depends heavily on the applications you run.

Start with say 4GB and ensure hot add/plug is enabled on the VM hardware so you can increase RAM if required without rebooting (assuming W2K8)

[CTSTech]

Josh Odgers wrote:

As you have 6 core CPUs, your optimal VM sizes are 1,2,3 and 6 as there can be divided into 6 with full numbers.

What do you mean by "optimal VM sizes"?  Do you mean each VM should have 1,2,3 or 6 vCPUs?

Thanks for your help.

[joshodgers]

In this sceanrio w/ 6 core processors, 1,2,3 and 6 are the optimal vCPU counts for a VM (assuming they need the CPUs)

For optimal CPU scheduling, the size of the VMs vCPUs should be able to be divided in full numbers with the physical coes of the processor.

4 core CPU

VM Sizes 1,2,4 vCPUs

6 core CPU

VM Sizes 1,2,3,6 vCPUs

8 core CPU

VM Sizes 1,2,4,8 vCPUs

and so on.

Hope that makes sense.

[joshodgers]

Here is an example of a VM which was overszied, and suffering contention, on a 6 core processor.

I resized it to 2 vCPUs (from 4) and look at the results

http://joshodgers.com/2012/07/25/vm-right-sizing-an-example-of-the-benefits/

[tHeiR1sH]

For the original poster and anyone who pops into this thread, following the recommendations listed in this thread so far, would be a bad idea.  The offered suggestions are fine for a normal VM.  TS/RDSH is a different beast and every VMWare advisor I have spoken with so far does believe me...until they manage one.

RDSH users demand a fluid desktop environment.  They're going to open multiple browser windows, search within Outlook, and even try to stream multimedia.  In my experience, you need threads...all the threads you can throw at the users to ensure the processing power is available.

One of my current RDSH servers is in a Dell R820 chassis.  Quad 8-core processors, 4-Disk SSD RAID10, 200+ GB of RAM all sitting on top of Windows 2012R2.  -That's 32 Cores.  With hyperthreading you have 64 cores.  At peak usage I have about 90 users running MS Office applications, Adobe Acrobat, and a ton of browsing.  With 32 cores applied I can only run about a 40 user load before the system pegs and stays at 95-100%.  At that utilization, odd things happen...icons go blank, don't refresh, Outlook search breaks, lots of things.  Then if I add cores it's much smoother.  I up it to 52 cores and the system will hit maybe 65% utilization but never more except maybe a blip.  I have 128GB of memory allocated and the system never grows the Pagefile.

The above is an example that you need to seriously think through what your users will use.  If I had the virtual resources I would assume 1 core per user.  $$$ is not unlimited so I settle for 1/2 core per user.  As far as memory, 1.5 GB per user is a safe starting point.  You could probably get by with 1GB per user.

Don't go cheap on disks.  I can't imagine running anything other than this SSD RAID 10.  Even with this HDD arrangement I still see spikes from the usual 0.02 - 0.10 disk queue up to 0.7.

Have fun and do a staged roll out.  You'll be happy you did it in increments and have the opportunity to iron out bugs.