VMware Cloud Community
teknogroup
Contributor
Contributor

Too much ballooning?

Hi,

I'm trying to understand why I'm seeing much ballooning on a VM: frequently ESX reclaims up to 7 GB from a VM configured with 12 GB and it is a burdensome process.

I think I have no much overcommit rate on the host and not much memory pressure but I'm not sure.

Some help to indagate?

Thanks

Reply
0 Kudos
21 Replies
AntonVZhbankov
Immortal
Immortal

Check your memory performance tab for this ESX.

Also check memory limit for this VM.


---

VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
teknogroup
Contributor
Contributor

ESX host has 32 GB of RAM; under "Performance" tab, I never see more than 28 GB of "Memory Granted" and 12 GB of "Memory Active".

The VM has no memory limits.

Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

From the COS off your ESX host run esxtop

In esxtop press "m" for memory statistics.

On the top line you can see your overcommitment rate.

On the second line you can see the status of your physical memory and how much memory is actually free

If you press "f" and then press "i" for MCTL (ballooning) stats, you get additional columns in the statistics overview.

MCTL?: is the ballooning driver active?

MCTLSZ: the size in MB that is actually reclaimed by the ballooning driver

MCTLTGT: the size in MB that is going to be reclaimed (targetted memory)

MCTLMAX: the maximum size that the ballooning driver can reclaim. Default is 65% of assigned memory.

HOWEVER you don't want any ballooning and should install more memory into your ESX host.

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
depping
Leadership
Leadership

and don't forget to press " S 2 " to refresh your information every 2 seconds Smiley Happy

Duncan

VMware Communities User Moderator

-


Blogging:

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Reply
0 Kudos
teknogroup
Contributor
Contributor

OK, thanks, during recent VMworld I discovered the marvellous world of "esxtop"!

Now I'd like to understand wyh ESX reclaims memory by ballooning if host is not under pressure.

Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

If you say there is no memory pressure on the host. The only reason I can come up with is that the VM has a memory limit which is lower then the assigned memory in the guest.

You mentioned earlier that the vm in question has no memory limit. Are you sure?

Maybe the vm had a memory limit in the future. I once saw that ballooning didn't went away after raising the memory limit. I had to reboot.

Did you try to reboot the vm?

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
weinstein5
Immortal
Immortal

Are you using resource pools?

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
teknogroup
Contributor
Contributor

No resource pools, no memory limits. Overcommit is 0.96.

I'd like to understand the reason why ballooning is triggered on a host. What counters do I need to look at?

Reply
0 Kudos
AntonVZhbankov
Immortal
Immortal

Memory granted != memory used. There is always some memory overhead due to architecture, ESX maintains a lot of data structures for each VM.

If you don't want to baloon specific machine, just set all its memory amount as reserved.


---

VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

Whenever you see ballooning, you are short of memory resources Period!

Overcommitment of 0.96 means 96%. So you gave away twice the memory you physically have installed into your box!!

Are you still sure you don't have memory pressure?

What do your physycal memory (PMEM) statistics look like in esxtop?

And how about TPS statistics?

The reason I think the ballooning is on that particular vm is that ESX will get the most gain from that vm because of the large memory size granted.

Setting the memory as a reservation wil only move the problem away to the other vms on the same host. If you do this my guess is that you see ballooning on the other vms instantly.

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
AntonVZhbankov
Immortal
Immortal

Arnim, you don't have to explain me things like 2+2=4, I already know that.


---

VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

I know that, as a top contributor, I don't have to teach you stuff.

I just want to contribute to answering the original question. Why is there ballooning inside my vm?

I should have responded to the original poster instead of you.

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
teknogroup
Contributor
Contributor

Whenever you see ballooning, you are short of memory resources Period!

OK!

Are you still sure you don't have memory pressure?

As said, looking at graphs under "Performance" tab, it seems that I never see more than 28 GB of "Memory Granted" and 12 GB of "Memory Active".

What do your physycal memory (PMEM) statistics look like in esxtop?

PMEM /MB: 32763 total: 272 cos, 415 vmk, 25694 other, 6381 free

And how about TPS statistics?

Ehm... where have I to look at?

The reason I think the ballooning is on that particular vm is that ESX will get the most gain from that vm because of the large memory size granted

OK, it is plausibile.

Setting the memory as a reservation wil only move the problem away to the other vms on the same host. If you do this my guess is that you see ballooning on the other vms instantly.

Now I just reserved some memory to this VM, according to VMware "Best practices" for VMs hosting Java applications.

I will keep under control the host to see if ballonning will be moved to other VMs.

Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

As Anton said before memory granted = memory used. So if you see 28MB granted it leaves you only 4GB free.

This is what I see from your PMEM statistics also. There is says 6381MB free.

These statistics is just a time sample. You should monitor is during the day.

So from my point of view with only 4GB free I would add memory to the box or add another server.

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
teknogroup
Contributor
Contributor

I understand... I focused my attention on "memory active" instead of "memory granted", thinking that "granted" was only an "high level mark"...

Thanks to all

Reply
0 Kudos
AntonVZhbankov
Immortal
Immortal

For. ex. you have 10GB RAM ESX. Minus 300MB to SC = 9.7GB available.

Let's say you want to deploy 30 VMs with XP, just clean install and nothing more. Each VM with 512 MB RAM, not counting overhead, this will lead us to 15GB memory granted. Minus 9.7GB = 5.3 GB overcommit.

You start VMs in 5 minutes interval, so it would be enough time fot TSP and all other processes to work. You'll have active for about 2 GB RAM, all other memory is free. And baloon will happen anyway, the only difference is balloon driver will reclaim unused, inactive memory first.


---

VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
avlieshout
VMware Employee
VMware Employee

Tekno,

I just had a similar phenomenon !

a vm was ballooning because I set a memory limit on it. That was exactly what I was expecting. so far so good.

Now let's put the memory limit back to "unlimited" by selecting the checkbox.

Ballooning didn't went away.! Reboot, Shutdown/restart no success. Whenever that vm was restarted ballooning kicked in instantly.

So I decided to change the memory limit from "unlimited" to value above the granted MB. Instantly balooning targetsize went down to 0.00 and balloon started to deflate.

When ballooning was down to zero again I checked the unlimited checkbox again.

VMware, Is this a bug?

So maybe you are suffering from something similar.

-Arnim van Lieshout

-


Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

Arnim van Lieshout Blogging: http://www.van-lieshout.com Twitter: http://www.twitter.com/avlieshout If you find this information useful, please award points for "correct" or "helpful".
Reply
0 Kudos
AntonVZhbankov
Immortal
Immortal

Arnim, you should submit SR.


---

VMware vExpert '2009

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
teknogroup
Contributor
Contributor

Anton,

I followed your reasoning but my "memory granted" is below physical RAM! I have 32 GB of PMEM, overcommit is 80-90%, but "memory granted" reached max 28-29 GB.

In your example, you have "memory granted" > RAM.

I'd like to understand WHEN balloning is triggered and where I have to read that host is under memory pressure.

Reply
0 Kudos