Memory usage and metrics

[markvr80]

I know there are old threads on this that I've read, but I still would really appreciate if someone could confirm a few things:

I have a host with 20Gb RAM running approx 15-20 VMs that have 2GB allocated to them. This is obviously over commiting the RAM almost 2:1.

The host permanently has a yellow or red alert symbol for the memory usage, which averages around 18Gb+ according to the"Memory Usage" under the "Resources" box on the Host Summary screen.

The VMs have a "Host Mem - MB" values between 500Mb and 1800Mb.

Take one typical VM for example that is has a reading of 1843Mb, the "Guest Mem - %" is 8%and on the Summary tab for the VM, the "Guest memory usage" is 160Mb. Looking at the Performance tab, "Memory Balloon" is 0.

OK, sorry for the long setup but the devil is in the detail.

My question is why does this have so much memory allocated to it, if it is using so little (8%)? If it is just the case that there is no demands on the Host memory, and so ESX hasn't needed to reclaim memory, then why does the host alert on memory usage? How am I supposed to know if the host is actually under memory pressure, and I need to power off some test/dev VMs or take action?

I've read a lot of threads/kb articles on memory on these forums, and understand the swapping/ballooning etc but nothing seems to match up with the scenario above! ?:|

[kjb007]

Remember the host doesn't really "reclaim" memory. It can balloon a vm to use that memory, but that causes the memory usage in the vm to go up, but that memory is being made available to service other vm's. The host only responds to the vm's request for more memory, and provides what is requested. When the vm no longer needs that memory, it just stops using it. Since it thinks it owns that memory, it doesn't have a way or a need to notify ESX that it doesn't need that memory anymore, so ESX can mark that memory as available again. That's why you typically only see the host memory usage increase, and not decrease. Hope that makes sense.

-KjB

VMware vExpert

Don't forget to leave points for helpful/correct posts.

[dominic7]

Which version of ESX are you running, and what type of hardware?

I know there is a bug with nehalems that have virtualized MMUs, and RAM usage displayed on the VC charts. The newest version of ESX ( In my case the 9/24 release of ESXi ) corrected the issue.

[markvr80]

It is ESX3.5U4 on Sun X4100M2s (AMD Opteron 2216). I don't think it is a bug, I think it is more ESXs metrics as there are a lot of forum threads about people confused over this.

Thanks, kjb007 - That makes sense and is my understanding of how it works as well. If the host becomes under genuine memory pressure then it will inflate the balloon in the VMs, forcing the VMs to use their own algorithms to swap out unused pages, and ESX can use the physical memory inside the balloon elsewhere. And if it then becomes very short of memory, it will then swap the memory out to vswp files as a last resort.

I'm just wondering why my host has a red alarm for memory usage, when the balloon driver isn't inflated (or very slightly), and the "guest %" is so low. Are the memory alarms fairly useless then? I'm trying to diagnose performance issues and thought it was memory but not so sure now.

[kjb007]

Remember vCenter is not designed to be a monitoring system. It is a management system, that provides some base level monitoring. It does not inspect balloon situations with its host memory usage alarm. It simply looks at the host memory usage, and if it is over a threshold, then you get an alert. ESX4 has a much larger set of triggers on which you can build alarms, but 3.5 is very lacking in the monitoring department.

I would suggest you look at 3rd party monitoring tools if you want a more practical/robust monitoring and alerting infrastructure. If you do a search, you can find a few open source options as well as some commercial ones.

-KjB

VMware vExpert

Don't forget to leave points for helpful/correct posts.