VMware Cloud Community
rvanthiel
Contributor
Contributor
Jump to solution

Resource pools

Hello -

Just want to double check my understanding on resource pools. If I take a very high level, simple approach on this - I could have two resource pools. One called High Priority with CPU and Mem shares set to High. I could have a second pool called Low Priority with CPU and Mem shared set to Low. These are the only pools, and both could be set to expandable reservations. In this case, I could add my high/low priority VMs respectively...and in the event there is resource contention, the pools would kick in the restraints. If there are no resource contentions, the VMs would continue on just as they are today.

Would this make sense in a small environment where there are no worries over delegation, charge backs to departments, etc.?

Thanks.

Reply
0 Kudos
1 Solution

Accepted Solutions
RParker
Immortal
Immortal
Jump to solution

Yes that is correct. This is how I do it (and I get a lot of flak for doing CPU affinity, memory assignments, etc..) but this is how you \*SHOULD* manage a server, because this is the proper way to know what is going on.

Leaving a server to manage itself is not a good idea, because in times of crisis, as you have already eluded to, you won't know which VM's can have priority. Setting up pools ahead of time will eliminate these problems.

So I agree with your pools. I do mine a bit more granular, because I divide the machine up into 4 pools, but if you have high priority VM's and low priority, then if that's what works for you, excellent!

You are setting up good values from the start.

View solution in original post

Reply
0 Kudos
19 Replies
rvanthiel
Contributor
Contributor
Jump to solution

Or in this case, should I not even be bothering with resource pools at this point and just rely on DRS to bounce things around?

Reply
0 Kudos
biekee
Hot Shot
Hot Shot
Jump to solution

I think your understanding about resource pools is quite alright.

In your case you still could create different resource pools to make a difference between vm's which always should have the highest performance and between vm's which aren't that important.

bk

RParker
Immortal
Immortal
Jump to solution

Yes that is correct. This is how I do it (and I get a lot of flak for doing CPU affinity, memory assignments, etc..) but this is how you \*SHOULD* manage a server, because this is the proper way to know what is going on.

Leaving a server to manage itself is not a good idea, because in times of crisis, as you have already eluded to, you won't know which VM's can have priority. Setting up pools ahead of time will eliminate these problems.

So I agree with your pools. I do mine a bit more granular, because I divide the machine up into 4 pools, but if you have high priority VM's and low priority, then if that's what works for you, excellent!

You are setting up good values from the start.

Reply
0 Kudos
juchestyle
Commander
Commander
Jump to solution

Hey Man,

Read this thread, it may help you to understand Resources before you go off and configure them thinking you understand how they work exactly. And yes, you are on the right track!

http://www.vmware.com/community/thread.jspa?threadID=73411&tstart=20

Respectfully,

Matthew

Kaizen!
rvanthiel
Contributor
Contributor
Jump to solution

Thanks everyone!!

Reply
0 Kudos
RaRaSysBoomBah
Contributor
Contributor
Jump to solution

It is incorrect to say that the shares kick in when there is resource contention. The share values are factored in during the resource entitlement calculation. The shares are in play whether there is contention or not.

I find it is best to use resource pools sparingly. Use them to cap resource utilization, but not to allocate resrouces. You can get yourself tied up in knots.

Reply
0 Kudos
RParker
Immortal
Immortal
Jump to solution

"It is incorrect to say that the shares kick in when there is resource contention. The share values are factored in during the resource entitlement calculation. The shares are in play whether there is contention or not. "

WRONG!

They \*DO* kick in during resource contention, that what shares are FOR, specifically.

A share of resource ONLY gets awarded to the highest share amount, and lower priority shares only get theirs, WHEN they are available. When there are ample shares of resource for ALL vm's ALL will get what they need. When they are low, or in jeapardy, lower shares will starve until there is enough to give. Otherwise, the shares make ZERO difference during times of high resource availability.

When resources are low, the SHARES make the difference. That's how they work.

your undertstanding of Resource Pools is flawed, you should ALWAYS use resource pools, not just to limit resource, they help control and manage VM's better. That's how they work, they help to allocate resource, you reversed the method of thinking about resource pools.

Reply
0 Kudos
RaRaSysBoomBah
Contributor
Contributor
Jump to solution

Look dude, you are wrong. The share values are in play always, not only when there is resource contention. In 2.5, that was the case, in 3.0, the share values are used when calculating the resource entitlement value.

Resource Entitlement

Resource entitlement of a VM or a resource pool is a measure of how many resources it deserves. ESX Server calculates both memory and CPU entitlements for the various VMs and resource pools. We shall only discuss the CPU entitlements in this document. CPU entitlements are calculated in concrete units (MHz). As a result, there are a fixed number of entitlements available on the host (corresponding to the host capacity) as a whole and those are divided among the resource pools and VMs based on their reservations, shares and limits. Once the entitlement has been calculated it is used by the CPU scheduler to give VMs and resource pools CPU resources in proportion to their entitlements. Note that if a VM or resource pool is idling and not utilizing its entitlement, the resources corresponding to those entitlements flows to other VMs and resource pools in the system.

The following rules (I will refer to them as entitlement rules in the rest of the document) determine the calculation of entitlement.

1. A VM or resource pool is always entitled to its reservation

2. A CPU entitlement can never exceed the limit, even if there are unutilized CPU entitlements available.

3. Entitlements are granted preferentially to the resource pools and VMs which have the least entitlement / shares ratio. This is because the VMs or resource pools with more shares are entitled to more CPU resources. In other words, we try to equalize the entitlement / shares ratio of all the VMs. This handing out of entitlements to VMs and resource pools is done in an iterative manner, where CPU entitlements are handled 1 MHz at a time, to the most deserving VM or resource pool.

SO...share values can affect VM performance even if host resource utilization is low and there is no resource contention. I have seen with my own eyes.

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

You are wrong.

Read the thread Dave suggested - especially the posts from puneetz.

Reply
0 Kudos
RaRaSysBoomBah
Contributor
Contributor
Jump to solution

I am not saying that shares don't resolve resource contention or that they don't work as advertised. What I am saying is that even if there is no visible resource contention, host CPU < 50% MEM < 50%, share values can still cause VM to perform poorly. Case in point, our cluster has four child resoruce groups with custom share values, 1000, 1000, 2500, 5500. The idea being that dev/test servers get fewer slices than pre-production and production. We had a VM sitting on a host that was crawling. The host resource utilization was low, CPU around 30% and mem lower. Guest operating systems looked fine from taskmgr. My last thought was shares b/c there were plenty of resources available, there was no contention. The server was in a resource pool with a share value of 1000. I moved it into the production pool, and performance issues cleared right up. The issue was the shares, the server was not getting anything scheduled because it's weighted value was too low. So again, as the document I provided supports, the share values are taken into account by the scheduler during scheduling, it does not just use shares to break a tie.

Reply
0 Kudos
RParker
Immortal
Immortal
Jump to solution

Note: Virtual machines proportionally benefit from extra resources when some allocations are

underutilized. Also, resource allocations degrade gracefully in overload situations.

Adding virtual machines to a system or modifying share allocations within a system changes the

total shares and each individual virtual machine's fractional allocation. Shares are used to

allocate the resources available after the maximum and minimum allocations have been

allotted. While proportional shares provide additional flexibility for resource allocation, the

underlying subtlety of this feature can lead to non-intuitive behavior and poorly configured

systems.

Fine, if I am wrong, it's only with regard to SHARES, but during times of contention. The rest of if I am right.

Resource pools:

! Flexible hierarchical organization . You can add, remove, or reorganize resource

pools or change resource allocations as needed.

! Isolation between pools, sharing within pools . Top-level administrators can

make a pool of resources available to a department-level administrator. Allocation

changes that are internal to one departmental resource pool do not unfairly affect

other unrelated resource pools.

! Access control and delegation . When a top-level administrator makes a

resource pool available to a department-level administrator, that administrator can

then perform all virtual machine creation and management within the boundaries

of the resources to which the resource pool is entitled by the current shares,

reservation, and limit settings. Delegation is usually done in conjunction with

permissions settings, which are discussed in the Introduction to Virtual

Infrastructure.

! Separation of resources from hardware . If you are using clusters enabled for

DRS, the resources of all hosts are always assigned to the cluster. That means

administrators can perform resource management independently of the actual

hosts that contribute the resources. If you replace three 2GB hosts with two 3GB

hosts, you don.t need to make changes to your resource allocations.

This separation allows administrators to think more about aggregate computing

capacity and less about individual hosts.

! Management of sets of virtual machines running a multi-tier service . You don.t

need to set resources on each virtual machine. Instead, you can control the

aggregate allocation of resources to the set of virtual machines by changing

settings on their enclosing resource pool.

They still contradict what he said. Resource pools are useful for more than just "shares"

Reply
0 Kudos
RParker
Immortal
Immortal
Jump to solution

You are still wrong about using Resource pools sparingly. They do much more than simple share management.

And you are giving out wrong information. I was ONLY wrong about SHAREs (I know this was different in ESX 2.5), but the rest of Resource Pools still hold true.

Reply
0 Kudos
oreeh
Immortal
Immortal
Jump to solution

They still contradict what he said. Resource pools are useful for more than just "shares"

I should have mentioned that I meant the shares Smiley Sad

Reply
0 Kudos
RaRaSysBoomBah
Contributor
Contributor
Jump to solution

I would argue that your arguement for the use of resource pools is subjective. You can not say that you are right or wrong, it is just the way you like to do things.

Agreed?

You can go back to customizing your myspace page now. Thanks for your in put.

Reply
0 Kudos
RParker
Immortal
Immortal
Jump to solution

I am in class RIGHT NOW. I brought up this EXACT post and question.

I am NOT, repeat, NOT wrong. YOU are in fact wrong, ALL (Dave, puneetz, and Oreeh) of you. SHARES ONLY come into play when the resources are 90% and above used. PERIOD.

There is NO argument. The instructor AND another VM technical representative say that VI3 works EXACTLY as I stated originally.

They told me to say that if you don't believe me, you should take a class, because it's obvious YOU don't understand how VI3 works.

SHARES are for CONTENTION and when resources are LOW, PERIOD.

We even raised the scenario that if there are PLENTY of resources, there is NO reason for the ESX server to divy up resources by share, if there enough to fulfill the request. It's like you have a pool of money, and you had enough for everyone, what purpose would it make to be stingy or only give money to certain people? It's not, and it doesn't. Shares are for times of priority when there ISN'T enough to go around.

This concludes the argument, and if you have any further discussion, you can take a VI Install and Configure class.

Reply
0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

I checked out one of the VMworld presentations yesterday, one of the best ones in my opinion, and they basically confirmed the same thing. Once it becomes available to the general public I would highly recommend checking it out.

TA21 Understanding "Host" and "Guest" Memory Usage

The presenters (Kit Colbert) also recommended this session...

IO50 VI3 Resource Management and DRS - Performance Use Cases

Reply
0 Kudos
xylog
Enthusiast
Enthusiast
Jump to solution

I think the confusion here springs from your definition of resource contention. Most people look at the average load on guest and host and say - Oh my server is running at 10% there is no resource contention. The problem is that this is an average. Peak utilization can exhuast CPU resources within that interval and shares will kick in even though it seems from looking at average that they will not. This is why shares affect performance even when there is not on "average" resource contention. You need to do some common sense arithmatic. If you have 10 vm's and only two CPU cores then there will likely be instantaneous contention for processing even though the overall average looks fine. So shares in this case can have a large affect during normal operation. If however you have a dual core 4 way with 8 processing cores and you run only 6 one vCPU vm's then there will likely be no CPU contention and shares will have little effect.

In my experience most shops load each physical CPU with many multiple vCPU so shares will usually have a perceivable affect on VM performance. Dont take my word on it though every environment is different and every workload is different you really need to do your own benchmarking to test the limits of what works. Shares can be a powerful tool, but they can also create alot of headaches if used without proper planning AND testing. You cant just read the docs and expect it will work for you in the real world.

Reply
0 Kudos
RaRaSysBoomBah
Contributor
Contributor
Jump to solution

How old are you? You sound like some whinning idiot on an MTV reality show. Shares do come into play when you are using DRS. Again, when using DRS, share values are taken into account when the resource entitlement algorith is run. If you are running stand alone ESX, you are correct. Does that make you feel better you gigantic whinning ass?

Reply
0 Kudos
esiebert7625
Immortal
Immortal
Jump to solution

Insulting and flaming other users is strictly against the Terms of Service of these forums. Please cease this immediately and post in a professional manner.

Eric Siebert

User Moderator

Reply
0 Kudos