optimize vCPU, vCore, vRAM

[thomascatty]

Hi everybody,

Just got my vmware ESXi5 essentials licences for my 2 dell servers hosted in Peer1 in London.

in short, their specification (for each) is :

- 128 Go RAM

- 2 x CPU Intel 2,5Ghz 6 cores ==> 12 cores ==> and probably 24 with hyperthreading.

- 2x 2To storage

The use : Technical Analysis based soft on financial markets (lots of data to manage from Thomson Reuters)

I chose to buy huge of RAM 'cause mysql cluster I'im using is consuming a lot (in Memory DB)

On each server, I'll have 4 VM, all running Debian 6 squeeze.

- On the first one, "just" a Webserver acting as a loadbalancer (Oracle Webserver iPlanet 7) : for loadbalancing and failover session

- On a second, a Java Application Server (Glassfish 3.1.2.2 : hosted all the java files for the soft and linked to the loadbalancer)

- On a third one, "just" one manager for mysql cluster 7.2 (no data)

- On the fourth one, the data node with the data, RAM consuming.

Without VMware I would have reserved approximatively :

- 4 Go RAM for the first VM (loadbalancer)

- 20 Go RAM for the second VM (Java Application Server) + the Java Application Server shoud support a load too (2 vCPU ??)

- 4 Go RAM for the third VM (mysql manager)

- 100 Go RAM for the fourth VM (mysql nodes) + I know the cluster needs at least 8 cores to run the best way (vCPU ?, VCores ? both ?)

So now, I would like to use the best values as vCPU, vCores ...and take advantage of Transparent Pare Sharing, Ballooning, swap..

I know that the only fact to use the same OS is good if I use some templates after the first completed install, isn't it (for T. P. Sharing) ?

But I don't exactly what to do with the vCPU, vCores...

I read about best practices but... your help will be more than welcome !

Thanks in adavance  for your tips, Dream Team !  😉

[jrmunday]

There  is unfortunately no right answer to this question without testing, and adapting based on the results of your tests. What we can hopefully offer is some guidelines on how to approach this benchmarking.

Start small by allocating the minimum number of vCPU's and RAM that you feel is appropriate, use the performance charts, esxtop and perhaps 3rd party tools to analyse the performance and increase the resources to match your requirements (don't forget to increase the load to cater for spikes I resource requirements). With regards to sockets / cores, it shouldn't make too much difference to performance but could be factored into licensing costs, especially where applications are licensed per socket.

Check you NUMA size, and try and keep your biggest VM within these boundaries for performance. Also make sure that you are on the latest build as there is a fix included in this which addresses a NUMA scheduling issue and this makes a huge difference to performance, and in specific reduces %RDY time drastically.

You also want to avoid ballooning and swapping if you can, so if you are seeing this then you will need to review your memory allocation and usage. With only 4 guest and 128GB RAM though, I doubt this will be an issue. But then again, if you estimate that your MySQL boxes need ~100GB RAM then this will need to be monitored carefully.

So in short, understand your requiremets and build an infrastructure to match these.

[meistermn]

Hire a consultant from software vendor  http://www.veloxum.com/

[RParker · Accepted solution]

Sounds like a very terse answer, but that is precisely what needs to happen.. someone needs to help and give you proper answers AFTER they analyze the environment.

[thomascatty]

hi jrmunday and thanks for quick answer,

To start with : YES it would be kind of you to give me me some guidelines for benchmarking. 😉

Sure there is no "good" answer, but I was looking for tips to achieve my goals, what you started to give me.

what is "esxtop" ?

I'm quite surprised by your mind about sockets and cores. I thought there was a real difference and different type of results when using more

vCores or more vCPU.

To be honest I didn't understand this paragraph about NUMA size : could you tell me more ?

"Check you NUMA size, and try and keep your biggest VM within these boundaries for performance. Also make sure that you are on the latest build as there is a fix included in this which addresses a NUMA scheduling issue and this makes a huge difference to performance, and in specific reduces %RDY time drastically."

I didn't understand why you said that "I also want to avoid ballooning and swapping if you can, so if you are seeing this then you will need to review your memory allocation and usage."

Sorry for the misunderstandinf but, as everybody I suppose, I want to take advantage the most of all the technologies : Transparent Page Sharing, Ballooning, Swap ... if possible  😉

A last question on this post would be : You know I want to allocate about 100Go for my data node box but this is to anticipate the coming size of my database in some months (one year maybe). It won't be like that at the beginning.

So here is my question : Is it very bad for performance to reserve so muvh vRam although it could be useless at the beginning ?

my own answer would be "no" if it is not at the cost of the others boxes ?

So I must set up all of my boxes in prod and test ! 😉 (it works in my DEV environment but with small DBs ...)

Thanks a lot.

[Lessi001]

Hello Thomas,

esxtop is a command line tool to do analysis of performance on your ESXi hosts.

Maybe this articles will help you:

www.yellow-bricks.com/esxtop

http://www.running-system.com/vsphere-5-esxtop-quick-overview-for-troubleshooting/

A good article about NUMA you can find here:

http://virtualtoddsbigblog.blogspot.co.at/2012/02/numa-performance-considerations-on.html

Regards

Andi