VMmark 1.0 HTML results - Time Synch problem

[rolandt]

I cannot obtain results using VMmark v1.0 with 1 tile. When I run the results compiliation script "tilescore2html.sh" after completing a run I get the following "Error: could not resolve start-time. Time on clients not synchronized".

I am using 1 tile so 1 client. How can there be a "clients not synchronized" problem with a single client? What is causing the problem and how do I resolve it?

A second question - what requirements trigger the generation of the "Score_1_Tile_Test.out" results file at the end of the run? The results file is NOT produced when I use RUNTIME=3600 (1hr) instead of the default RUNTIME=10800 (3hr).

thanks.

[kimono]

You need to make sure time is synched on all clients and all workload VMs. I found after vmotioning all workload VMs to another host and rerunning the test, it failed for the same reason because the host had incorrect UTP time, and all the VMs were synching to the host, so they got it all wrong.

I've ended up setting all VMs to sync to mailserver0's NTP port, and all CLIENTS to sync using domain time synching with w32tm /config /syncfromflags:DOMHIER /update.

Question2 - Only a compliant test run (RUNTIME=10800 (3hr) will generate this file.

[rolandt]

I know the clients and VMs need to be synched. But I expected them to be in synch since for this initial test I am using all VMs (including the client) on the same ESX host. And VMware Tools is set to "time synch with host" for ALL[/b] VMs (again including client). Alas, that apparently does not work well enough.

I will look at trying your solution of synching everything to tile0 mailserver0.

[kimono]

I did have it working sync to host, when I was only running one tile, but it went bad after the vmotion to another platform and some VMs lost their time. It wasn't until I logged on to every VM to check that I discovered this.

have fun

[kimono · Accepted solution]

I've emailed the VMWARE VMMARK person about consequences using NTP instead of Tools sync, as it does contradict what I read in the VMWARE Time Keeping document. Will post back what I hear. I would have preferred to use Tools sync but had to go NTP as a work around.

[psmith2006]

The tilescore_wfmt.pl script is missing an edit to the error message which

should read (as it does in vmmark/tilescore.pl): "Error: could not resolve start-time. Results data missing from one or more *.wrf files or time on clients not synchronized"

So given that you are doing a 1-tile test, the most likely issue is that at least

one of the tests have failed. If inspecting the *.wrf files does not

provide a clue to help solve the problem, then re-run the test with

DEBUGFLAG=1 in the VMMARK.CONFIG file and the zip the Results_

directory and send it to vmmark-info@vmware.com

We'll check it out and see if we can't get the issue resolved.

[psmith2006]

RE: A second question - what requirements trigger the generation of the "Score_1_Tile_Test.out" results file at the end of the run? The results file is NOT produced when I use RUNTIME=3600 (1hr) instead of the default RUNTIME=10800 (3hr).

The harness calls tilescore.pl - only if the test is the regulation runtime

(10800 secs). However, if you run a shorter test you can get very unofficial

measurement values by using the optional arguments shown in the usage:

\# ./tilescore.pl

Usage: ./tilescore.pl -t tilescore.csv

So a: ./tilescore.pl -t 1 -w 10 -m 15 will generate unofficial values

assuming data for all workloads is available for this 55 min. measurement.

Note: These values MUST NOT be publically represented as VMmark scores since

they would be non-compliant with the VMmark run and reporting rules.

[psmith2006]

Just so everyone understands that you can NOT generate Official VMmark results

using a client that is a VM on the system being tested. Measurement timestamps

are taken on the client(s) not on the workload VM's to ensure accuracy of the

measurements. If you have multiple clients their time and date needs to be in

sync with the other clients within a few seconds.

The benchmarking guide includes recommendations for synchronizing time

on the clients and on the workload VMs.

[rolandt]

Yes, I know all clients being on seperate physical systems is one of several requirements for a valid submittable VMmark result. Publication is not my objective with these tests.

My Time Synch solution.

I setup and am using my Prime Client (called "control", which is yes, yet another seperate physical system) as an NTP server. All clients (client0, client1, etc) and all VMs use "control" as their only NTP source. And, of course VMware Tools Time Synch is disabled in all VMs.