CPU Reservation

[gdesmo]

I have a vm that consumes alot of CPU, 4500mhz curentlly. Under resources-cpu settings the reservation is 4000mhz. And the "unlimited" check box is checked under limit. It seems to respond better with reservation set at 0 with unlimited checked.

Is it better to leave the reservation at 0 mhz? Thanks

[jjohnston1127]

Setting a reservation will guarantee that the VM will get that many CPU cycles if they are available in the resource pool IF THE VM NEEDS them. It does not mean that it will by default automatically consume 4000Mhz of CPU cycles. If you leave reservation to 0 and set to unlimited, it will consume as many resources as it needs (that are available) and will not be limited.

If you are having issues with it consuming too much, you may try setting a "limit"

[gdesmo]

So setting a 0mhz reservation allows it to use more than a 4000mhz reservation setting? That doesn't make sense to me.

Especially when having "unlimited" checked. I would think this would override any reservation value and give it as much as it wants.

[jjohnston1127]

Remember, "reservation" means GUARANTEE. If you set the reservation to 0, it means that at times when an ESX host is consuming a lot of its resources, that particular VM is not guaranteed any CPU cycles.

If the reservation is set to 4000Mhz, at times when the virtual machine will need 4000Mhz, it is guaranteed to get it, no matter if they are available or not. The ESX host will steal CPU cycles from other virtual machines to fulfill the guaranteed reservation of 4000Mhz.

A "limit" says that the VM cannot consume any more CPU than what is allocated. If it is set to unlimited, it can consume as much CPU that is physically available on the ESX host that it will need.

If you want to limit it to 4000Mhz, set the limit to 4000Mhz and the reservation to something lower.

[gdesmo]

Yes exactlly, so I would think a reservation of 4000mhz would perform just as well as at 0mhz. Not worse, which is what I am seeing. At least the vm would be guaranteed 4000mhz at any time. I set it back to 0mhz and the cpu in task mgr goes way down.

DRS gives me a headache....

[samugi]

CPU Reservation will have no affect on performance unless your virtual machines are in contention for exhausted processor time. So if your esx host is under heavy processor load and you have a VM with say a CPU reservation of 1GHz your other virtual machines will have degraded performance while the VM with the reservation will consistently get 1Ghz of CPU time . If the resource pool the VM is in doesn't have 1GHz to give the vm and your reservation is expanded enabled the host will attempt to 'borrow' CPU time from the resource group ABOVE the one your VM with the reservation is in.

Reservations can be seen as a way to try an ensure a base level of performance. ("I reserve 1Ghz of cpu for this virtual machine")

Limits are a way to control how much resources a vm (or resource pool) can use. So if you have a VM that you know is a cpu hog and you want to make sure it doesnt use more that 500Mhz.. you would set a limt of 500Mhz. Even if nothing else is running on the host and there is plenty of CPU available that VM will only get 500Mhz.

You can also use shares which basically set the priority of resources. A VM or resource group with a higher priority(or shares) will get more of the resource in contention senarios than a VM or resource group with lower priority (or shares)

Reservations and shares only come into play when a resource is in contention (3Ghz being requested by multiple virtual machines when only 2Ghz is available)

Limits are alway in effect if set, so if you have 16 GB of RAM available a vm with a 512Mb limit with get only 512Mb even if it needs more.

Hope this helps.

[samugi]

Also,

Task manager is not a reliable tool to measure CPU for a virtual machine. Use the metrics afforded by Virtual Center to indicate the resources (cpu, memory, etc) a VM is using.

[gdesmo]

I forgot to mention I have no resource pools created. But DRS is enabled on my cluster.

So what your telling me is what I was thinking. A reservation of 4000mhz vs 0mhz should not degrade vm performance. But it seems to have done just that. Based on VI performance measurments as well as task mgr.

I have gone back to 0mhz reservation, unlimited limit and custom shares of 1329. I did not set the share value. I do not think it is affecting my performance?

[JaySMX]

Exactly, you should not see degraded performance when a VM actually has a reserved amount of CPU. Like said above, all this only matters when there is contention for resources. If you are able, try migrating other VMs so that this one VM is the only one running on a particular host and see if you see the same results. You can leave it at 0MHz reserved, the only time that may cause issues is if the VM actually needs 4000MHz and it is not available within the resource pool, without that reservation it may not be able to get the CPU cycles it needs.

[petedr]

Setting the reservation to 400 mhz is guaranteeing the CPU not setting a limit. Putting it to 0 is just saying don't reserve any CPU. Like was said setting to 0 or 400 should not effect your VM performance. If you set a Limit then that could effect the performance depending on how low the setting was.

Below is copied straight from the Resource Management Guide which explains Reservations.

Reservation

Reservation specifies the guaranteed reservation for a virtual machine. The server

allows you to power on a virtual machine only if the CPU and memory reservation is

available. The server guarantees that amount even when the physical server is heavily

loaded. The reservation is expressed in concrete units (megahertz or megabytes). When

resources are not used, the ESX Server host makes them available to other virtual

machines.

For example, assume you have 2GHz available and specify a reservation of 1GHz for

VM1 and 1GHz for VM2. Now each virtual machine is guaranteed to get 1GHz if it

needs it. However, if VM1 is using only 500MHz, VM2 can use 1.5GHz.

Reservation defaults to 0. It is a good idea to specify a reservation to guarantee that the

necessary CPU or memory are always available for the virtual machine.

[samugi]

Ok so you agree that a reservation should not affect your performance but you say it is. I guess my next question would have to be is this VM operating in a bubble. Are there any other VM's running? What are their resource controls set to and what is the overall CPU load of the host.

Your base question was is it better to have a reservation of 0? My general answer would be yes leave the reservation at zero and utilize shares to control which vms or groups of vms get the resources in states of contention. My only caveat to this would be unless the host in question MUST maintain a certain level of performance. If you want to say that this vm needs 4500Mhz regardless of what other vms are doing then set a reservation. but if you just want to say that if the processor is under contention prioritize this vm higher than other then leave reservation at 0 and just set a higher reservation. This is just my personal prefernce I am sure other may feel differently and that circumstances may make alternative approaches more appropriate.

I tend to like one method and not mix them so I will usually just set share levels on resource groups. If I do have a VM that is high visability and sensitive. I will set a reservation on it. But I don't micromanage reservations on multiple VMs

Im not sure why setting a reservation to 4000 made your vm performance degrade from when it was at reservation 0 with no processor contention (or even with) I'd like to see a copy of the performance chart.

Also are you making these changes then rebooting the VM or just leaving it running. Do not reboot the VM as the inital stabilization, memory normalization etc will throw your metrics off. Let the vm run for a while, stabalize then make changes to resource controls and watch the affect the changes have.

[gdesmo]

So to give a vm all the resources it would like I could check "unlimited" under cpu and memory? And set the shares to high under both cpu and mem?

And if the vm has been given 1 gig of memory. I see you can adjust the limit to above what the vm has been allocated. Would checking "unlimited" alllow it to use more than 1 gig?

I did notice after a vmotion to another host my vm which was pretty much idle in mhz spiked after the vmotion. And remained spiked.

I have gone thru all vm's on this host and set shares to normal. Some of them were @ custom which was a lower value than normal. And then I set all too no reservation. And checked "unlimited" on cpu and mem. Overall host utilization does not apear to be an issue. It's using about 50% of 24ghz and 50% of 32 gig. I have not re-booted any of the vm's after changing resources.

Thank you

[jjohnston1127]

Correct. If you check "unlimited" for CPU and RAM it will allow the virtual machine to use as many resources that it needs (that are available out of the resource pool).

If the VM is allocated 1GB of memory(when building the machine), that is simply telling the guest operating system: "Hey, You have 1024mb of memory in your configuration." With the limit set to "unlimited" it can actually consume more than that if it needs it.

Setting the CPU and RAM shares to High will give it priority over other virtual machines on the box to use the resources that it needs. Look at it this way: If you give it 8000 Shares for CPU and all the other VMs have 2000 Shares, it has 4x the shares, so think of it as getting 4x the I/O and Cycles it needs. It will essentially get 4 I/O cycles per every 1 that is granted to the normal VMs.

This is all relative based upon how you actually configure it though.

[samugi · Accepted solution]

This is not totally accurate. When you set the memory on the VM it sets the virtual machines "hardware" visable memory which will be a portion of the the ESX host memory. The operating system will not exceed this regardless of what your limit is set to . So if you set memory in the virtual machine hardware to 512MB, and the ESX host has 8GB even if the VM is memory starved and needs more memory and the limit is unlimited or set to 2GB the VM will only get 512MB. In this condition the VM will have performance degradation. This has nothing to do with VMware but is how the operating system works. You can of course shut the VM down and increase the memory in the VM settings which is essentially the same as adding more memory to the virtual machines "physical" characteristics. This is true of Windows virtual machines. There may be guest OS's that can have dynamically increased memory that I am unaware of. But I'm sure they do exist the ability to increase the memory is performed by the VM settings not the Limit.

The statement on shares is correct. however, it is important to realize, and it's been stated here several times, that shares and reservation only have impact when the resource is in contention by multiple virtual machines.

[jhanekom]

gdesmo, I realise it's a bit late in the thread to be asking this, but something I'm missing is that you've not explained quite how you determine that performance is worse after adjusting the reservation setting?

[samugi]

It is detailed in the thread. At first he stated he was using perfmon. I told him perfmon can not be accurately used to measure VM performance. He said he also used the performance metrics from VC. I'd like to see the output (screenshot) both with and without reservation.

[gdesmo]

I ended up realizing I was being affected by:

I think after understanding this bug I am seeing more predictable results. Sorry for the confusion...

[samugi]

Ah... I considered this being part of the issue but related to rebooting the virtual machine after making changes not DRS. When a VM boots it will take time for its performance and memory utilization to stabalize. They will initially be high. This is the same for any Vmotion activity which is why you would see it with DRS.

Glad you got it resolved.

[jhanekom]

scpierce / gdesmo: I'm still not quite clear on exactly how he determined he had a problem, but glad that he managed to solve it

Thing is, the original post (and several subsequent posts) claimed different behaviour between setting a reservation of 0Mhz vs. setting a reservation of 4000Mhz. By definition, there should be no difference in performance unless there is contension, which the OP has stated there isn't.

My question was mostly just around how the difference in performance was measured - this was never explained. Higher utilisation in VC/VI means higher utilisation, not necessarily reduced performance...

Still, I learned something useful from that link - thanks gdesmo!