By now everyone knows that guest performance metrics are not reliably in virtual machines (Guest-based Performance Measurement). Slight fluctuations in very small time measurements and a lack of knowledge of the hypervisor's activities produces misleading numbers. Windows' performance monitoring tool, Perfmon, suffers from the same problems. However, Perfmon remains highly valuable for virtual machine performance analysis.
The performance analysis methods document that was published early in 2008 provided tips on esxtop-based performance analysis. To record esxtop data, page 159 of the resource management guide describes running esxtop in batch mode. But on ESX Server 3.5, running esxtop in batch mode with all counters enabled results in an incredibly large CSV file that cannot easily be parsed. But Perfmon can help with this process.
esxtop was constructed so that its CSV-formatted batch output file can be readily consumed by Perfmon. This means that Perfrmon can be used for two key activities:
- Quickly analyzing results.
- Generating smaller CSV files of a subset of the data that can be more easily consumed by other analysis tools (such as Microsoft Excel.)
Quick Results Analysis
esxtop's batch output CSV file can be opened and viewed in Perfmon with the following steps:
- Transfer the CSV file to a Windows sytem.
- Launch Perfmon (Run: "perfmon".)
- Right click on the graph and select "Properties..." from the drop-down menu.
- Select the "Source" tab.
- Select the "Log files:" radio button from the "Data source" section.
- Click the "Add..." button.
- Browse to and select the CSV file created by esxtop and click "OK".
- Click the "Apply" button.
- Optionally: reduce the range of time over which the data will be displayed by using the sliders under the "Time Range" button.
- Click "OK".
Once the data has been loaded into Perfmon you may select ESX performance counters and display them using Perfmon's graphing system just like normal Windows performance counters. Refer to the Perfmon documentation for instructions on doing this.
Pruning CSV File Data
Because so many people use Microsoft Excel to analyze performance data but its row and column limitations are quickly exceeded when generating large esxtop batch files, a means of removing unneeded data will assist analysis. Once Perfrmon has been loaded with the esxtop data, it can be used to generate smaller CSV files that can be easily consumed by Excel. Follow these steps:
- Start Perfmon and load the esxtop batch data as described above.
- Have the Perfmon graph display the data of interest (that you'd like to import into Excel.)
- On the graph, right click "Save Data As..."
- In the popup box, select "CSV" as type and save the file.
This will save only the counters that were displayed in the graph. By iteratively selecting subsets of counters and saving off individual CSV files it becomes possible to quickly build performance graphs in Excel using esxtop batch output data.