VMware Cloud Community
thiteshew
Contributor
Contributor

High Virtual Memory usage on ESX Server

Hi all,

One of our ESX servers is practically out of virtual memory (swap). I'm new to Linux/ESX but I used this article to do some troubleshooting: http://rimuhosting.com/howto/memory.jsp

Doint a free -m shows swap of 541 total and 540 used.

When I do a ps aux, this process is using almost 19% of memory:

root 7971 0.0 18.5 165544 49852 ? S Apr25 48:38 /opt/vmware/aam/bin/ftPerl /opt/vmware/aam/bin/ftmainrule.pl -domain vmware -a (gaddr=4/3.6[0])

This looks like some kind of perl script? I've checked one of our other ESX servers and this process isn't running at all. Can anyone please provide some insight into what this process is and what would be the best way to get some of the virtual memory back. Thank you.

Reply
0 Kudos
10 Replies
gxl24
Enthusiast
Enthusiast

The perl script is calling something for Legato Automated Availability Manager. Do you use any Legato products?

How much physical memory and how many VMs do you have running? Swap should only be used when physical mem is low. I understand this process is taking 19%, is the remaining being used by VMware?

thiteshew
Contributor
Contributor

We don't have any Legato products, as far as I know.

The physical memory seems fine. The Vi3 client shows that the server is using approximately 16 of its 32 gigs of RAM. There are 20 VMs on this server, which includes our production Exchange server, but none of the VMs seem starved for resources.

Reply
0 Kudos
weinstein5
Immortal
Immortal

I assume you are running your esx host in an HA cluster - AAM is the technology behind HA so you do not want to do anything with that process - what we are really talking about here is the virtual memory of the service console - things that will use memory include AAM as you pointed out, back up agents, management agents, vms with attached CD ands floppies, and VM Consoles created through the VI client or web access - I would look at those first - also you can increasee the service console memory from 272 (the default) to a maximum of 8090 MB - but if you do this you will also need to enlarge the swap space used by the service console -

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
weinstein5
Immortal
Immortal

See my other post AAM is used for VMware HA - also if the VMs seem to be unaffected I would not worry about increasing the size of the service console memory but I would take a look at ways of reducing the load like disconnecting or removing cd-roms from the vms, use an RDP client to interact with the vm rather than the vm console - just to name a few

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
thiteshew
Contributor
Contributor

Thanks for the response Weinstein. I found that one VM had a CD ROM device connected to an ISO and one VM had a floppy connected. I've disconnected the devices and will see if that helps the memory.

Reply
0 Kudos
thiteshew
Contributor
Contributor

There has been no change in the virtual memory usage. The server is still only using about half of its available physical memory, but is using 539 of 540 megs of swap file.

Reply
0 Kudos
weinstein5
Immortal
Immortal

The other items that will consume memory in the service console are if the primary method of communicating the guest o/s is through the vi client's vm console or web access - so if you have users who are doing that to the other 20 vms that will take memory. Another issue is if people are running applications inside the service console,and I am sure the other 19 vms still have cd roms and floppy disks - even if they are not putting to local resources the service console still uses resources to present those to your guests - the affests of the service console runningo ur of memory include poor vm console performance, unable to start vms, poor communication to VC - so if you are not seeing those I would not be concerned -

also what does the free command show for physical memeory used?

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
Reply
0 Kudos
thiteshew
Contributor
Contributor

free -m shows 250 of 262 used. This is the same for our other ESX server.

Reply
0 Kudos
thiteshew
Contributor
Contributor

Two of our other ESX servers are showing high (over 90%) virtual memory usage.

We use NetVigil to monitor these servers via SNMP. The Long Term History report shows that virtual memory has been steadily increasing since May, which is about the last time these servers were rebooted. Physical memory is not high on any of these servers.

Host.d logs have several messages relating to Memory Checker Current value exceeds soft limit .

Any ideas besides rebooting the ESX servers?

Reply
0 Kudos
arisaperstein
Enthusiast
Enthusiast

I found ftmainrule.pl to consume nearly 100% of CPU 0. I perfornmed the "Reconfigure for VMware HA" in VC and the CPU utilization went all the way down.

Reply
0 Kudos