Java JVM not using memory

[sroggen]

We're trying to fix a problem with Java running on ESX 3.02 with a SuSE 10 guest OS. On one of our server applications repeatedly crashed after about 20 minutes of running when we started it in a VM but it would run forever (> 4 hours) running the same guest OS on a physical machine. The message given at the abend is a fatal exception due to exhausting Java's heap. In both cases we use the same amount of heap, specified by a command line argument of 256 MB. We can get the VM to run without crashing if we give it unlimited memory, but it seems to expand to about 450 MB before it stops growing. Performance is much poorer on the VM than on physical.

I put together a web application using Apache Tomcat, Xalan and some documents to try to exercise the Java memory management aspects of the JVM. This application transforms documents from XML to XHTML and sends them back to the requester.

Performance on a physical machine (4 CPU/32 GB RAM) is relatively good. If we run the application with only -verbose:gc on the Java command line, we see the application consume up to about 800 MB of virtual memory with many (hundreds?) minor garbage collection occurring between major garbage collections. At each major garbage collection the heap gets reduced to about 100 MB and then grows linearly towards a maximum of about 800 MB before the major garbage collection occurs again.

The behavior on a VM (2 vCPU, 1 GB reserved allocated RAM) is completely different. The only command line argument given to the Java VM is -verbose:gc. In this case the heap only grows to 83.320 MB before garbage collection occurs. Minor garbage collections only occur early in the run with all garbage collection becoming major after only a short duration. The heap, as reported by the JVM, does not seem to grow. Occasionally we get an exception thrown that will indicates an out of heap condition.

Finally, if I increase the heap size to 256 MB on the virtual machine, we see it only expanding to a fraction of this value, to about 173 MB.

We're. stumped by this behavior. Does anyone have any ideas why we do not see the heap memory expanding to a larger value when running under VM?

[esiebert7625]

Are you sure your physical and virtual servers are configured identically? You're setting your inital JVM size to 256MB, are you specifying a maximum size also? Going from a 32GB physical server to a 1GB virtual server is a big reduction in memory, have you tried giving the VM more RAM?

[AlbertWT]

Please refer to this documentation http://www.vmware.com/resources/techresources/1087

but meanwhile, The following are general recommendations from VMware KB site:

Size the virtual machine memory to leave adequate space for the Java heap, the other memory demands of the Java virtual machine code and stack, and any other concurrently executing process that needs memory from the guest operating system.

Set the memory reservation value in the VMware Infrastructure Client to the size of memory for the virtual machine. As any type of Memory Swapping (physical or virtual) is detrimental to performance of JVM heap especially for Garbage Collection.

Determine the optimal number of virtual CPUs for a virtual machine that hosts a Java application by testing the virtual machine configured with different numbers of virtual CPUs at different times with the same load.

If you are using multiple Garbage Collector threads in your JVM, match the number of those threads to the number of virtual CPUs that are configured in the virtual machine.

For easier monitoring and load balancing, use one JVM process per virtual machine.

If your ESX Server is overcommitted, ensure the the Balloon Driver is running within the virtual machine so that memory is optimally managed.

Kind Regards,

AWT

[NuggetGTR]

wow you blew the dust off this post

[AlbertWT]

Sorry man (-_-)" hopefully I can still be helpful in giving the suggestion.