Windows VM all RAM stolen by driver-locked i.e. VMTools?

[Clarkington]

I am running ESXi 5.5.0 on a single host. I've discovered that a "memory" leak on my Windows 7 Enterprise VM was due to VMTools consuming nearly all the RAM as "driver-locked" I had no memory limit or reservation on the VM. After I uninstalled VMTools it could no longer consume the "unused" memory and performance is fantastic (no more page file thrashing) however I now don't have VMTool functionality on the VM.

Has anyone figured out a good workaround/resolution to this issue? Its online enough I'm not the only one with the problem. I was thinking of maybe doing a memory reservation for the VM, however I don't like the idea of reserving a chunk of the pool - that inefficiency is why I went with VMs in the first place.

Additionally, does anyone know how to check for driver-locked memory in Linux? I wonder if some of my CentOS VMs with "memory leaks" are experiencing the same issue as well.

[Clarkington · Accepted solution]

More Internet research has let me to discover that sounds like this is WORKING AS DESIGNED. I have over-committed the amount of physical memory in my host and VMWare is using its balloon driver to make each VM think it is low on memory so that it frees up memory marked for garbage collection so the ESXi hypervisor can know to then use those memory locations to provide memory to other VMs that need it.

The below blog post is excellent at explaining what is going on. The workaround is to set a reservation for your host:

http://www.vfrank.org/2013/09/18/understanding-vmware-ballooning/

When I look at the Performance tab for my ESXi host and select memory, the amount in Measurement Balloon is ginormous. This makes sense because all my VMs are running VMTools and they are all being ballooned so that they can all work together.

My next task is to determine what combination of memory allocation tools (reservation, priority, pool etc) will yield the best performance)