In my recent whitepaper on Java performance, I used tc Server 2.0 as the deployment platform.  The underlying web container for tc Server 2.0 is Tomcat 6.0.24. In the course of my tuning experiments, I gathered some interesting results about the performance benefit of switching to Tomcat’s NIO HTTP connector.  These results are not specific to the virtualized environment, and I thought that they would be worth sharing.

The Tomcat HTTP connector provides a connection from clients using the HTTP protocol to the web applications.   Tomcat has three different implementations available for the HTTP connector.  There are the default Java HTTP connector, the high performance Java NIO HTTP connector, and the native APR HTTP connector.  \

I compared the peak throughput of my application (Olio) for native and virtual configurations with a single CPU using the default HTTP connector and the NIO HTTP connector.  The results are summarized in the chart below.  Switching from the default HTTP connector to the NIO HTTP connector resulted in a 150% to 200% increase in peak throughput.  No other tuning was done to achieve this increase.

Note that Olio is a very network intensive workload.  At peak-throughput, the 1-vCPU VM was, on average, receiving 85Mbps and transmitting 161Mbps. The benefit of switching to the NIO HTTP connector would likely be lower for less network intensive workloads. 

I learned two important lessons from these results.  The most obvious was that if you are using Tomcat, you should not be using the default HTTP connector.  The NIO HTTP connector can give an impressive performance boost, and the configuration changes needed to enable it are trivial.  I did not try the native APR HTTP connector. 

The second lesson was that the performance overheads introduced by virtualization are trivial compared to the potential impact of misconfiguration and poor tuning.  The same lesson could be drawn from my experiments on the performance impact of maximum heap-size.  I also saw improvements from database and file server tuning that, while less dramatic, also overshadowed the virtualization overhead.  While this isn’t really a surprise, it is useful to keep in mind when trying to improve the performance of virtualized applications.

In my recent whitepaper on enterprise Java performance on tc Server 2.0 and vSphere 4.1, I discuss tests that I ran with native and virtualized configurations using 1, 2, and 4 total CPUs.  The setup that I used when I collected the published results reflected the results of some tuning experiments that aren't covered in the paper.  At the JVM level, the two main tunings that I investigated were the maximum heap size, and the impact of using large pages.  In this blog I will look at the impact of changing the maximum heap-size.

As I mention in the paper, the application that I used was the Java version of Olio, a Java web application that implements a complete social networking website.  When choosing my JVM tunings, one of the first things that I wanted to understand was how changing the maximum Java heap size affected the peak number of Olio users that could be supported by the application. I also wanted to understand how changing the number of CPUs affected the choice of heap size.

The chart below shows the change in peak-throughput as I increased the maximum heap size for the different VM configurations.  In order to avoid confusion with the published results, which reflect additional test-bed tuning, I normalized all of the values to the lowest datapoint.

There are a few interesting observations to be made about this data. There is clearly a tradeoff to be made between heap size and performance.  However, after a certain point, the benefit of larger heap sizes is limited. Depending on memory availability, it may worthwhile to give up a bit of peak throughput in order to save memory.  This is particularly true if reducing the heap size, and therefore the VM’s total memory size, allows you to avoid over-committing memory on an ESX host.  While memory over-commit works well for most workloads, Java applications tend to suffer when ESX has to reclaim memory from the VM.

It is also clear that the same heap size won’t work for all VM configurations.  In my tests I chose to use a maximum heap size of 2048MB for a 1-vCPU VM, 2560MB for a 2-vCPU VM, and 4096MB for a 4-vCPU VM.  If I had used 2048MB for all three configurations, I would have lost about 5% off of the peak-throughput in the 2-vCPU case, and a painful 35% in the 4-vCPU case.  Of course, the specific heap-size needed, and the impact of changing the VM configuration and heap-size, will vary for different Java applications.  However, the message is clear.  When moving from one hardware configuration to another, it is important to re-assess the choice of maximum heap-size.  This is true whether you are changing the VM configuration, or moving from physical on one system to virtual on another.