Memory Ballooning

[tku703]

Hi all,

I'm new to this forum and kinda have a noob question on memory ballooning.  Now i know that memory ballooning is one of ESX RAM reclaiming techniques and it balloons a guest OS RAM and swaps it to the guest OS swap file the host created, is that right?

Say I have a VM with 2GB of RAM allocated, ballooning occurs and balloons to 1GB.  I guess my question is, this 1GB thats ballooned, is this reserved for the VM (1GB the host can't touch) or this 1GB moved off the VM?  Im just trying to understand how ballooning works and comparing that with the information under the "Resource Allocation" tab of a VM in Vsphere client.

Hope this makes sense.

[Busbar]

Please keep in mind that memory configuration depends on what is reseved and what is maximum memory configuration for the VM.

for the sack of simplicty, if the machine has reserved 1 GB memory and 2 GB memory max, then it will always gurantee the 1 GB but it can (on demand) go for 2 GB memory, ESXi will monitor the memory usage of the machine and if it is not hot pages will baloon them to claim the memory back to the host, to be used with other VMs.

hope that it is clear now.

[sparrowangelste]

"By default the balloon driver is used to reclaim idle memory."

"When the balloon inflates the GOS will first assign all “unused / free”  pages to the balloon driver. If this is enough it will stop. If this  isn’t enough the OS will decide which pages it will page out  until it  reaches its threshold."

http://www.yellow-bricks.com/2010/03/11/mem-idletax/

so that 1gb your vm has, if it has more idle memory or swappable mem, it can still be swapped out.

[rickardnobel]

tku703 wrote:

it balloons a guest OS RAM and swaps it to the guest OS swap file the host created, is that right?

No, not really. If the VM is forced to swap it will do so only to its own internal swapping space, like the pagefile.sys in a Windows Server. The VM is not aware of the swap file created by the ESXi host.

[tku703]

Wow, everyone, thanks alot all the replies.  So, correct me if i'm wrong here:

1.  When VM1 needs RAM, the ESXi host finds another VM (VM2) not using RAM

2.  Ballooning occurs on this idle VM2

3.  The host sends this reclaimed RAM from VM2 to VM1.

is that right?  So basically, the ballooned RAM (say 1GB using the example above) is unusable to the VM2 and it just there for show?  Also, where is the host swapfile for each VM located?

Thanks again!!

[sparrowangelste]

balloning should only occur if there isnt enough ram free.

but if there isnt enough free ram in the host, then yes 1-3 would happen...

for vm2 the ballooned ram would be pshycial  host mem it no longer has, but the vm still thinks it has 2gb.

the swap file is normally located with the vm, vswp unless you specify it is somewhere else

[Busbar]

AFAIK balooning is done dynamicly even if there is free memory to reclaim cold pages, correct me if I am mistaken

[rickardnobel]

Busbar wrote:

AFAIK balooning is done dynamicly even if there is free memory to reclaim cold pages, correct me if I am mistaken

The ballooning is actually only done when the host is under some memory pressure, as in something like only 4-6 % free physical memory. If not under that condition it will not do any ballooning (except a configured Memory Limit at a certain VM).

The ballooning process could not easy reclaim cold pages either, since the balloon driver must "hold" that pages by force inside the Guest Operating System, to prevent the OS from putting other data there.

[Busbar]

Thanks mate for elaborating that.

[proden20]

Ballooning and swapping are two different things.  Ballooning forces Windows to release unnecessary RAM address space on the host and force that data to pagefile.sys, thereby freeing up physical RAM for other VM's to use.

VMWare Swapping occurs when either a MB limit set by the administrator is reached or ballooning isn't solving the host's constraint problem.

[tku703]

So if a VM is ballooning its RAM, what data is being swapped?  I thought ballooning occurs on a VM thats running idle and only ballooning un-used guest RAM?  Is this data being swapped to the guest OS swap or host swap file?

[Busbar]

the swapping will occur with the VM is trying to use more memory and there is limits configured or there are no memory to supply by the host, then swapping will occur.

[rickardnobel]

tku703 wrote:

So if a VM is ballooning its RAM, what data is being swapped?

Hopefully no internal swapping needs to be done, but that the Guest Operating System could release pages used for cache or similar.

tku703 wrote:

I thought ballooning occurs on a VM thats running idle and only ballooning un-used guest RAM?

It does not happen when a VM is idle, but when the ESXi host is under memory pressure. The VMkernel has also no idea of what will be "ballooned" since that it is the Guest Operating System who will decide.

tku703 wrote:

Is this data being swapped to the guest OS swap or host swap file?

If the Guest is forced to swap it will swap only to its internal swapping file, e.g. pagefile.sys. The guest has no way of accessing the external Vmkernel swap file.

[asvidra]

Try to Explain with a simple scenario(if i am wrong please correct my understaning)

Lets take an ESX Server has three Virtual Machines(VM1,VM2,VM3), each using a RAM of 8 GB.

VM1 uses its maximum of 8GB RAM and now it needs more speed if any resource is try to add to it.

VM2 uses just 3GB out of 8GB, so now VM1 use the remaining RAM speed from VM2 to store its new resource. this is called as Memory Ballooning.

If VM2 needs more RAM, then the VM1 allows the RAM Speed it taken from VM2 and it came to hung state.

"Utilizing the unused memory is ballooning".

Message was edited by: asvidra

[rickardnobel]

asvidra wrote:

Lets take an ESX Server has three Virtual Machines(VM1,VM2,VM3), each using a space of 80 GB.

VM1 uses its maximum space of 80GB and now it needs space if any resource is try to add to it.

VM2 uses just 30 GB out of 80GB, so now VM1 use the remaining space from VM2 to store its new resource. this is called as Memory Ballooning.

That is not really the way it works. If VM1 has 80 GB of RAM configured it has no way of ever using more than that. Ballooning can not take any RAM from any other VM to give VM1 in this specific case.

[adrianych]

Sorry for bringing up this old thread as I found some in-consistancies in some of the theories above...

I have a Physical server with 72GB RAM and 4x 4core CPU.

- running 5 VMs with total 12vCPUs (2 vCPUs x4 VMs + 4vCPU x1 VM) and 38GB RAM (4GB x2 VMs + 8GB x2 VMs + 6GB x 1 VM)

- one of the 5 VMs is running Win7 64bit (4v CPU and 6GB RAM)

The VM that I noticed with ballooning is the Win7 64bit VM (running Acronis).

When the backup jobs started I noticed the performance chart of the VM (approx 45 min) :

- RAM usage was a U shaped, starting from 6GB , dropped to 3GB after 15 min then back up to 5.5GB

- RAM Balloon was a n shaped, started from 4GB, dropped to 3.5GB but remained there till almost when RAM usage went 4GB, then dropped to 0.

- CPU chart shows violent swings between 10% and 30%

So what was the ballooning for ?

[proden20]

I have seen this happen when the guest is paging heavily and can't keep up with the application's request for RAM (particularly under SQL.)

The basics:

- Make sure you do not have any limits set for CPU & Memory

- Make sure you have the latest VMWare tools driver installed

- Make sure you have 1.5x the RAM allocation (9GB) assigned to your pagefile (or select "System Managed")

- Ensure that your pagefile is not sharing the same disk spindles as the backup target

- Ensure any memory leak type bugs are resolved in Acronis by updating your software

- Ensure that you are using the Recommended memory size for your backup application and workload

- Determine if you can throttle Acronis memory consumption

See if any of those help.

[rickardnobel]

Adrianych wrote:

So what was the ballooning for ?

Based on the amount of RAM in your host and the vRAM attached to your VMs you should really never see any ballooning, no matter if the VM is paging internally or not.

Could you verify if the VM has any memory limit configured?

[proden20]

In our situation, we found that someone had forced sqlserver.exe priority over the Windows kernel, which forced an epic battle between OS and Application. The end result was the vmkernel trying to alleviate an erratic guest's memory requests with vmkernel swap and ballooning.

Once the DBA's reversed what they had done, all was calm and the ballooning and swapping stopped.

Strange, but true.

[rickardnobel]

Dennis Procopio wrote:

In our situation, we found that someone had forced sqlserver.exe priority over the Windows kernel, which forced an epic battle between OS and Application. The end result was the vmkernel trying to alleviate an erratic guest's memory requests with vmkernel swap and ballooning.

It seems incredible strange that the balloon or vmkernel swap should get involved in any way because of how the internal memory handling is done. The principle in general is that the vmkernel has no, and needs no, knowledge of what the guest place inside its own memory and should not have any opinions if the RAM is being heavily used by either 10000 instances of notepad.exe or a misconfigured SQL server. What the os and the running processes does in RAM "should" be invisible for the vmkernel, until either a vmkernel RAM Limit is reached or if being close to the amount of physical ram on the host.