VMmark 2.1 results in QOS score of 0 on the mail server

[Rishi2012]

We were looking for some quick assistance as to why our QOS  score for a 2 tile test is resulting a 0 score for the mail server? We are trying to identify if this is a configuration issue on our part that is causing the issue. We have met all the requirements set forth in the VMmark 2.1 benchmark guide from an infrastructure standpoint so we believe the quality is there. I am attaching our latest score results. Any assistance would be greatly appreciated.

[jpschnee]

Hi,

Typically, this is due to a configuration issue.  When I reviewed your results I see almost as many task exceptions as there are tasks dispatched.  I would recommend that you run Loadgen manually, pointing it at the config file under c:\vclient0\vmmark2-mailserver-0.xml.

Remember not to change any of the populated values except for run time.  Doing this should help uncover what may be causes all the exceptions.

[Rishi2012]

We have a 2 tile benchmark where the prime client is acting as the client for the 1st tile. Our client 1 is hosted in a different cluster the our 2 tiles of VMs. When I run the loadgen test manually on both client0 and client 1 it successfully completes. We have not altered our vmmark config file beyond the run time and adding the variables unique to our enviorment.  Do we have a next step to figure out why QOS is 0 for the mail servers in our score file?

[jpschnee]

When Loadgen completed, did it report any exceptions?

[Rishi2012]

Do you mean under the load generator status on the results page? If so, the count for the number of task exceptions for client 0 and client1 reads as follows:

Client 0 (Prime Client)-1648068

Client 1-1648068

looking at the task exception statistics, I see the source of all the exceptions is the following message:

Microsoft.Mapi.MapiExceptionLogonFailed

I verified the login crednetials in our vmmark2 config file are accurate. is there another source for this logon failure?

[jpschnee]

There are many reasons why you might be getting this error so I'm not sure.  How did you verify that the login credentials used were correct?

[Rishi2012]

I verified by using those credentials to login to the mail server . i was able to logon successfully. Are there some other top hitters that causes this message?

[jpschnee]

I'm unaware of any other main reasons for getting this error within VMmark2.  Can you zip up your entire results directory from your last run so I can review?

[RebeccaG]

Hi, can you give the error description for Microsoft.Mapi.MapiExceptionLogonFailed? For example, Unable to make connection to the server or Unable to open message store.

[Rishi2012]

Where do I get the details of the error message Microsoft.Mapi.MapiExceptionLogonFailed? i do not see anymore details on the LoadGen status screen other than error I posted.

[RebeccaG]

On the client, go to C:\Program Files\Exchange Load Generator and look in the log files LoadGenInit* and LoadGenSim*.

Thanks, Rebecca

[Rishi2012]

Thanks. Looks like it is the message store:

Microsoft.Mapi.MapiExceptionLogonFailed: MapiExceptionLogonFailed: Unable to open message store. (hr=0x80040111, ec=-2147221231

[RebeccaG]

This is most likely a permissions error. I would echo jschnee's request for you to zip the results directory from your problem run and attach to this thread, please. Another place to look for more informative errors is the Event Viewer on your Mailserver VM, under Windows Logs > Application.

[Rishi2012]

Thank you both for your assistance thus far. I have attached the files from our last run. Looking at the application event log on the mail server VMs. We did notice the the following MS exchange IS error. Which should be remedied by the following KB:

http://technet.microsoft.com/en-us/library/bb232050(v=exchg.80).aspx

Once that setting was checked, the mail databases would mount. However, after checking the status of the setting after they were mounted, it was unchecked again. It may take new off-line backup of the database to preserve this setting through a reset, or it may take something else?

[RebeccaG]

Hi, could you create a zip file with all the files from your run and attach the zip file please?

It should not be necessary to check "This database can be overwritten by a restore". To clarify, when that box is not checked, are your databases listed as unmounted?

If you would like to try checking the box and running again, follow the directions on page 152 of the VMmark Benchmarking Guide to delete and remake the Microsoft Exchange database.

[Rishi2012]

Sorry for my delay in response but I havezipped up my results file. In addtion to the issue, i ahven't found a solid answer as to how the QOS score is calculated? What does a 0 really mean? Our other VMs have a score assigned to the QOS.

[RebeccaG]

A 0 means the Mailserver QoS metric is not being measured at all because the workload is failing. All Exchange actions attempted on the Mailserver are failing. LoadGen is responsible for reporting each action's latency. Loadgen is reporting the latency for all Exchange actions as 0, therefore, the harness also reports it as 0. However, the failed tasks are being recorded as 'completed' although they are not completed correctly, which is why it appears that work is being done.

I have a couple of questions for you:

It's not necessary to check "This database can be overwritten by a restore". To clarify, when that box is not checked, are your databases listed as unmounted?

I would recommend following the directions on page 152 of the VMmark Benchmarking Guide to delete and remake the Microsoft Exchange database. Also please check there is enough disk space on your Mailserver VM.

I saw very odd VC and hostnames containing a large number of v, c, l, d, and X's in your results files. These names were inconsistent even within the same file. Are you running on a non-English version of Windows? VMmark has been tested only on English windows and is not supported on other languages. Or, did you intentionally obfuscate the names of your hosts, LUNs and VC in these files before uploading them?

Thanks,

Rebecca

[Rishi2012]

It's not necessary to check "This database can be overwritten by a restore". To clarify, when that box is not checked, are your databases listed as unmounted?

Yes they were unmounted, and that is why the team attempted to check this box.

I would recommend following the directions on page 152 of the VMmark Benchmarking Guide to delete and remake the Microsoft Exchange database. Also please check there is enough disk space on your Mailserver VM.

We have attempted the step to delete and remake the DB from pg 152 but were unable to have success with this QOS score being greater than 0. Also our Mail server VMs meet the storage and memory requirements stated the Benchmark guide.

I saw very odd VC and hostnames containing a large number of v, c, l, d, and X's in your results files. These names were inconsistent even within the same file. Are you running on a non-English version of Windows? VMmark has been tested only on English windows and is not supported on other languages. Or, did you intentionally obfuscate the names of your hosts, LUNs and VC in these files before uploading them?

yes I intentionally obstructed the names of the servers, VMs, and user name and passwords. I probably should have followed a standard to prevent confusion. There that is why you see this inconsistency.

Thanks,

Rishi

[RebeccaG]

The databases should be mounted on their creation. One reason for the databases not to mount is that there is too little disk space on the VM's C drive. Even if you have made your VM with correct size disks, the virtual hard drive of the VM may become filled up with logfiles if you are encountering a large number of exceptions while running the workload, as you have been.

In the Mailserver VM, please check that there is a reasonable amount of free disk space (more than 1 GB) on C:\, E:\ and F:\. Could you tell me how much space remains on these drives, please?

If this is not clearly the problem, could you delete the databases as described on page 152 and recreate the databases. As soon as you create the databases, observe whether they are mounted or not. If not, please check the Event Viewer log  (Windows Logs > Application) to see if there's an error recorded there from the database creation. If any errors are reported during the database creation process, please report those here too.

Thanks,
Rebecca

[Rishi2012]

On our 2  mail server VMs, i see this for the available space that is somewhat consistent.

C Drive-2.44 GB

Database E Drive-25.5 GB

Backup F Drive-1.56 GB