ESXTOP Memory monitoring

[esxinoob]

Hi

I try to understand the ESX memory resource management.

I try to intepret the attached statistic:

1. 65Gb memory are available, 11Gb of them are not in use
2. Although memory is still free, it gets memory of VMS over the balloon driver

3. The esx host is swaping files from memory to disk (!)

Can someone explain why the ESX host does ballooning and swaping, even there is 11GB unused memory?

May I intepret it wrong?

The ESX Host is part of a cluster (6 hosts). The other hosts does show the value of MEMXL and SWAP = 0.

Would be glad if someone could help me

[lenzker]

usually esxi starts to baloon when it leaves the high state and gets into soft state ( depending on the build-in RAM, usually when 99% are consumed)

So since you are still in the high state, the only explanations might be the use of resource pool/virutal machines with configured limits. Do you have somewhere limits configured?

[esxinoob]

I just checked the ressource allocation tab - there are some machines with memory limits.

Hm, so these machines are running on limit and the host won't allocate more ram because of the set limits...

Now ballooning takes effect  - does it mean that the balloon driver takes ram of other vms?

What are the consequences of ballooning / swapping?

[lenzker]

the limit configured defines the maximum amount of memory the virtual machine can consume in the physical memory, the difference will be swapped/balooned.

" does it mean that the balloon driver takes ram of other vms?"

> balooning takes place to reclaim the consumed ram of VMs, in short it takes ram of VMs to free it up for other VMs.

I recommend you to watch the vmworld session: understanding virtualized memory performance (http://www.youtube.com/watch?v=YKaUtoQrLjo&feature=plcp ) or the VMware optimize & scale class 🙂

[grasshopper · Accepted solution]

In your case, the VMs being hurt are the ones with the limit set (they are hurting themselves of course).  In general the other VMs that are not misconfigured will be fine.  However, all the additional paging can have a neggative impact on your shared storage where the vmdk's live (that's where the guest swaps to).

As you may have noticed, if you give a VM 8GB of RAM and set the limit to 4GB, it will swap 4GB to the guests local disk (.vmdk).  If you then increase the memory to 12GB and leave the 4GB limit in place, then the guest will balloon and swap 8GB.  This is easily seen in the vCenter performance charts.

If you are a PowerCLI user, you can quickly find and fix these.

Find VMs with Memory Limits:

Get-VM | Get-VMResourceConfiguration | where {$_.MemLimitMB -ne '-1'}

Fix VMs with Memory Limits:

Get-VM | Get-VMResourceConfiguration | where {$_.MemLimitMB -ne '-1'} | Set-VMResourceConfiguration -MemLimitMB $null

Note:  After fixing these (either manually or via PowerCLI), you should power cycle the VM to enjoy the benefits.

General Balloon Driver Info (from Resource Mgmt Guide):http://pubs.vmware.com/vsphere-4-esx-vcenter/index.jsp?topic=/com.vmware.vsphere.resourcemanagement....

PS - Often the source of this misconfiguration is a template with a limit set.  So be sure to also manually check your templates to prevent this from sprawling in the future.

[esxinoob]

I'm a bit confused, because esxtop shows that a vm is ballooning, which hasn't a limit set 😕

its a VM with 4gb RAM allocated, limit set to "unlimited".

How do you explain that phenomenon?

[grasshopper]

Ensure that the VM has been power cycled and VMware tools are healthy.  If this is a one-off anomaly, you may consider un-register / re-register the VM.  Of course, it could also be valid ballooning

[lenzker]

it might be that the VM has balooned in the past....the baloon driver only deflates if the VM access the balooned memory

[esxinoob]

I restarted the correspronding virtual machine. After that, balloon has shown a value of 0 - seems to be fixed now.

Thanks for your help!