Resource Pool Reservation Question

[twoton]

We've been using vSphere for a long time and this is the first instance where we have looked in to using resource pools to manage cluster resources.  What we are trying to do is pool funds from several different groups into a hardware purchase, then split computing resources amongst those groups equally but allow others to grow beyond what their guaranteed fair share if the resources aren't in use by other groups.  And if other groups begin to need those resources back, we want vSphere to give back the "lended" resources seamlessly without causing any VM power on issues etc.

I'm fairly sure resource pools set with reservations equal to a specific group's allotment is the way to go here.  As I understand, resource pool reservations do not behave like VM reservations and even if I were to collectively reserve 100% of the cluster resources across all of the pools, one resource pool could still grow beyond it's reservation as long as there was no contention.  When contention does start, extra resources given to a resource pool will be taken away, but only down to it's guaranteed reservation. 

Is my understanding of this all correct?  Is there anything I am overlooking that may cause issues, or a better way of accomplishing this task?  Thanks in advance. 

[twoton]

Sorry, should have added vSphere 5.0 u2 across the board. 

[weinstein5]

Welcome to the Community - Actually Resouce Pool are very similar to VM reservations in that they they gaurantee a certain amount of resources to the VM or Resource Pool when there is contention -so your description is correct -  One thing to keep in mind as you power on VMs in the RP it will take a portion of the reserved resource which by default is 1/2 of the assigned memory if no reservation is set - so if you try to power on VM when there is not sufficient reservation space available the VM will not power on - one way to avoid this is to make expandable reservation is permitted this way if there is still resource available in the parent pool (in your case the cluster) will be used

[twoton]

Thank you for your response, need to pelt you with question now though so brace yourself!

If I did an expandable reservation, wouldn't that prevent vSphere from taking resources back from a RP in the event that other RPs start to consume more and need their reservations back?  Would there even be resources that the expandable reservation could claim if I have already divvied up 100% of my cluster resources evenly as reservations across the RPs? 

And sorry, I should have said that RP reservations do not behave exactly like VM reservations.  I thought that they would not prevent VM's from powering on since the VM itself does not have a reservation.  Just that the VMs inside a RP would be guranteed the resources inside the pool and if for example the cluster was at 100% usage and every RP was using it's maximum amount of reservations, the VM's inside a RP would fight for resources the same way they would if they were in their own cluster with no resource pools?

[weinstein5 · Accepted solution]

This is the way I used to teach about reservations - think of a restaurant that has 100 chairs - as the VMs power on they have have a certain number of chairs that they have reserverd - if the vm needs more chairs it will get more chairs as long as those other VMs aren't using the chairs they reserved - no what happens if all the chairs are reserved - the vm will not be able to come into the restaurant (power on) even if there are empty chairs - if expandable restaurants reservations are enabled a call is made to the upstairs dining room (parent pool) to see if there are chairs available - this works the same way for memory and cpu except memory has the ability to use the per vm vmkernel swap file andto continue my analogy if all the chairs are reserved then and a vm shows up they might be seated in the patio which can make converstaion difficult (affects performance).

When a vm is powered off (leaves the restaurant) it will release its reservation making room for more VMs.

[twoton]

Thanks, it's funny how much you forget when you don't use things regularly.  After reading your response and going through various articles and a little test setup, all of it is coming back to me now.  I was thinking reservations were what I wanted here but I really just want to define shares for each RP.  As long as the share value for each RP is identical, each group can have any number of machines configured for whatever totals of CPU/mem they want.  If one RP is working extremely hard while the rest of the cluster is idle, they will be allowed to consume a high amount of resources.  However, if contention rises and resources need to be distributed evenly to the other RP's, it will be.  I can verify this by looking at the RPs resource allocation tab and checking the worst case allocation for the VM's; The total worst case allocation for all VM's in an RP will be equal to their portion of the cluster resources and a VM in a RP or even an entire RP that is consuming large amounts of resources will not cause other RPs to starve for resources. 

[weinstein5]

That is correct - shares satisfy you needs -

[twoton]

Just a follow up on this...

While the shares accomplished what I was looking to do, I did notice that when a RP was relatively idle overall but other VMs in different RPs were making use of the extra resources, things would be a little sluggish on a VM in the idle RP at first, until resources were taken back from the overactive VMs in the other busy RPs.  What I decided to do was create a small reservation on each RP so that some resources would always be reserved exclusively for use by a RPs own VMs.  This allowed responsiveness on the machines to always be snappy unless there were other VMs in the particular RP that were consuming large amounts of resources.  The ultimate goal here is to allow resource sharing but not allow one RP to impact the operations of another.  This setup seems to be the best way to accomplish that. 

[weinstein5]

Check out Duncan Epping's site Yellow Bricks (yellowbricks.com) he has done a number of articles about Resource Pools and resource manangment

[twoton]

He does have good stuff, but I looked there before posting here and didn't see any posts directly related to what I was doing.  I did find a post on Frank Denneman's site (Resource pools memory reservations - frankdenneman.nl) regarding resource pool reservations.

This is a quote from his post:

"RP reservations are used when children of the resource pool uses memory and the system is under contention. Reservations are not wasted and the resources can be used by other virtual machines. Be aware, using and reserving are two distinct concepts! Virtual machines can use the resource, but they cannot reserve this as well if it is already reserved by another item."

That blurb is what got my thinking originally about using reservations at the RP level.  I went back and tried this again and it does seem to work although not fully as I would have expected.  When I originally tried this I was looking at some misleading metrics that made me think it wasn't working.

So right now my test set up is this (no other VMs/RP's in the cluster besides what I have listed and testing) and I am just testing with CPU resources right now, all mem settings have been left to default.

Parent RP (to act as the mock "cluster")-

-6000 mhz cpu limit

-no reservation

-expandable reservation enabled

3 Child RPs (Child1, Child2, Child3)-

-no cpu limit

-2000mhz cpu reservation

-no expandable reservation

-2 VMs in each RP with 4vCPU's each

If I set up a process to peg the cpu on child1vm1, it will consume above the 2ghz reservation for the RP which is what I want it to do, however it maxes out at 3.5 ghz.  Not entirely sure why.  I would have thought that with all other VMs being idle and no other significant CPU resources being consumed, the CPU of child1vm1 would max out somewhere close to the 6ghz limit of the "cluster".

At this point I am again confused for a couple reasons.  First, why this is working the way it is and second, given that I have a couple different possible solutions, which would be the best way to go.