When does ballooning kick in?

[JoeSaulDearman]

Hi

can anyone tell me exactly when a host will start using ballooning to reclaim memory. I've been told that it is when total host memory usage is at 80% but I cannot find this any any official documentation. If any knows what settings are available to specify when this should kick in, that would also be very useful

Many thanks

[rsaha]

See attached doc.Hope it will help

[weinstein5]

I do not know what the exact threshold is but I have always understood that Ballooning kicks in when the vmkernel cannot satisfy the VMs need for memory so even at 80% utilization ballooning migh not kick in if all VMa are receiving sufficient memeory resources -

[a_p_]

It depends on the vSphere version as well as the memory in the host itself. I'm currently reading Duncan Epping's (http://www.yellow-bricks.com/) book "vsphere 5.0 clustering technical deepdive" which explains all aspects of memory reclaiming in detail.

By default the threshold for free memory on a host is between 6% and 1% (depending on the host's physical memory)

André

[mittim12]

AP is right on with his numbers.   This article , http://frankdenneman.nl/2010/06/memory-reclaimation-when-and-how/, is a great resource on when ballooning and swapping will kick in to reclaim memory.  

[a_p_]

My appologizes to Frank Denneman.

The book I mentioned before was written by Duncan Epping and Frank Denneman.

André

[stinklyonion]

as long as all VM's receiving enough memory, everything should be fine and ballooning will not kick at all even reaching 85-90%. but i know someone where ballooning kicked in while he is still at 70%, i dont really know what he did and what really happened. that will always depend.

[mittim12]

Well one example of ballooning kicking in early would be limits.  If you gave a VM 5 GB of memory and then turned out and set a limit of 3 GB on the VM.  I believe it would start ballooning.  

[a_p_]

You can have ballooning kicking in with only a low percentage of the physical RAM being used. E.g. with configuring memory limits or working with specific settings in resource pools.

André

[shishir08]

Memory reclaimation tecniques are kicked in when host is in high memory pressure. ESX analyzes its memory state, and depending on that the reclamation techniques will be kicked in.

ESX could be in three state:

High(6%)-ESX is not under memory pressure.(Transparent Page  Sharing is only active now)
Soft(4%)-In this phase ESX prefers to use balloning to statiate its demand,where it uses the non active pages to do the Ballooning
hard(2%)-In this phase It will start to swap to the disk and thus ESX will start to actively reclaim memory when it’s running out of free memory.
and low(1%) - Swapping-The performace will degrade enormously because swapping can swap  the active pages as well.

[JoeSaulDearman]

Thank you Andre. Is there somewhere in vShere where I can see the threshold for a host? I am using verison 4.0.0.

[JoeSaulDearman]

Shirshir08, thanks for you input - can I ask what is the difference between hard (2%) and low (1%)? From what you say it looks like swapping occurs at both of these states. Is there something more that occurs at the low state?

[a_p_]

I'm not sure whether you can see the configuration settings themselves in the GUI, however you can see some information about the current state using esxtop. Anyway, even though his may not exactly be the correct forum for this question, I moved the discussion from VMware ESX™ 4 to vMotion & Resource Management Duncan Epping sometimes visits this forum and I'm sure he can provide helpful information.

André

PS: If you rather want to have the discussion in the "old" forum, let me know and I'll move it back.

[shishir08]

At the ESX(2%)- Swapping and Balloning both will happen and at ESX(1%)- Swapping will happen.

Remember Ballooning initialy uses non active pages to Balloon but when Memory pressure  increases beyond some limit then in that case it will start using active pages to Balloon,where Swapping and ballooning throughput performance becomes the same.

Swapping can use any page to swap , it could be active/non active page.

[benny_hauk]

Thanks mittm12 for bringing up the memory limits being a potential reason.  I kept seeing balloning swooning upwards when VM would approach approx half if installed memory.  Didn't seem right based on the amount of available memory in the host so I went digging to see what else could cause ballooning other than low available physical memory.  Ding!  "Unlimited" wasn't checked in memory resources and there was a limit for some dumb reason for half the memory in the box.

Thanks again!

[S_Grimbuhler]

HI

An ESX uses 4 states to describe its memory.

memory states are :

- HIGH. Until 6% memory free. No action

- SOFT. Between 6% and 4%. Action = ballooning

- HARD. Between 4% and 2%. Action = ballooining and swapping

- LOW. Between 2% and 1%. Action = swapping

Normally it should never fall bellow 1% because at low state, RAM is not anymore used, only swap.

The esxtop (or resxtop) command shows the memory state.

I have to say this is for vSphere 4.x I dont know if anything changed on vSphere 5 (I guess no).

Hope it helps.

Stéphane Grimbuhler

Senior Virtualization & Storage consultant (VCP / VCAP-DCA)

VMware Instructor (VCI)

My Blog : www.virtualgeek.ch

[rickardnobel]

shishir08 wrote:

Remember Ballooning initialy uses non active pages to Balloon but when Memory pressure  increases beyond some limit then in that case it will start using active pages to Balloon,where Swapping and ballooning throughput performance becomes the same.

There is a difference between ballooning active pages and doing Vmkernel Swapping when it comes to performance, and that would be that forcing the guest to swap/page internally by increasing the balloon driver memory size - that the guest operating system still has to do the selection of what to page out to disk. The Guest OS will most likely swap/page the most uninteresting and least accessed areas and leave kernel files, device drivers, active services and similar intact in RAM.

When doing vmkernel swap it will just pick random pages "from the outside" and this will typically hurt performance more.