VMware Performance Community
mathieugont
Enthusiast
Enthusiast
Jump to solution

VMmark/Auction fails with RC=1

Hi David

Thanks for your advice.

I thus setup an NFS I have mounted on each cluster of my datacenter (same absolute path on each node). See the print screens below.

I had a close look the the property file, but I do not see how I can point my shared file-system.

Moreover, VMmark/Auction fails with RC=1 which refer to an API issue (see the messages bellow).

Any idea of what is going wrong? I am running out of ideas.

Thanks.

20190522-11:02:14 Info  Process: Client0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable doesnot exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:02:14 Info  Client0 : Get Guest Info did not run correctly

20190522-11:02:14 Info  Process: AuctionLB0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:02:14 Info  AuctionLB0 : Get Guest Info did not run correctly

20190522-11:02:14 Info  Process: localhost : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:02:14 Info  localhost : Get Guest Info did not run correctly

20190522-11:02:14 Info  GuestInfo Results : AuctionLB0 :: Failed : Errors 0 : ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:02:14 Info  GuestInfo Results : Client0 :: Failed : Errors 0 : ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:02:14 Info  GuestInfo Results : localhost :: Invalid : Errors 1 : ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:03:14 Info  Process: AuctionWebA0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:03:14 Info  AuctionWebA0 : Get Guest Info did not run correctly

20190522-11:03:15 Info  GuestInfo Results : AuctionWebA0 :: Failed : Errors 0 :ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:04:15 Info  Process: AuctionWebB0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:04:15 Info  AuctionWebB0 : Get Guest Info did not run correctly

20190522-11:04:15 Info  GuestInfo Results : AuctionWebB0 :: Failed : Errors 0 :ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:05:15 Info  Process: AuctionAppA0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:05:15 Info  AuctionAppA0 : Get Guest Info did not run correctly

20190522-11:05:15 Info  GuestInfo Results : AuctionAppA0 :: Failed : Errors 0 :ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:06:15 Info  Process: AuctionAppB0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:06:15 Info  AuctionAppB0 : Get Guest Info did not run correctly

20190522-11:06:16 Info  GuestInfo Results : AuctionAppB0 :: Failed : Errors 0 :ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:07:16 Info  Process: AuctionDB0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:07:16 Info  AuctionDB0 : Get Guest Info did not run correctly

20190522-11:07:16 Info  GuestInfo Results : AuctionDB0 :: Failed : Errors 0 : ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:08:16 Info  Process: AuctionMSQ0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:08:16 Info  AuctionMSQ0 : Get Guest Info did not run correctly

20190522-11:08:16 Info  GuestInfo Results : AuctionMSQ0 :: Failed : Errors 0 :ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:09:16 Info  Process: AuctionNoSQL0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190522-11:09:16 Info  AuctionNoSQL0 : Get Guest Info did not run correctly

20190522-11:09:17 Info  GuestInfo Results : AuctionNoSQL0 :: Failed : Errors 0: ErrorMsgs  : Warnings 0 : WarningMsgs

20190522-11:09:47 Info  Guest Info failures for the following machines: 10 machines : Client0 AuctionLB0 localhost AuctionWebA0 AuctionWebB0 AuctionAppA0 AuctionAppB0 AuctionDB0 AuctionMSQ0AuctionNoSQL0   Error Messages Client0= :AuctionLB0= :localhost= :AuctionWebA0= :AuctionWebB0= :AuctionAppA0=:AuctionAppB0= :AuctionDB0= :AuctionMSQ0= :AuctionNoSQL0= :

pastedImage_0.png

pastedImage_1.png

pastedImage_2.png

Tags (2)
Reply
0 Kudos
1 Solution

Accepted Solutions
dmorse
VMware Employee
VMware Employee
Jump to solution

Hi mathieugont

Sorry you're still having issues.  A few things I want to point out:

  • It looks like you have created a cluster for each ESXi host, and the first cluster, ClusterESXi001, has 1 host, but all of the tile 0 VMs on it.  This will not work for multiple reasons:
    • The VMmark3 benchmark is designed to vMotion, storage vMotion, etc. VMs from 1 host to another within the same cluster.
    • The benchmark is designed to only work with a single client ESXi host cluster (ClusterESXi005 in your case), and a single server ESXi host cluster (which you have broken out into ClusterESXi001,2,3,4).
    • Creating shared storage such as NFS, while it may allow the benchmark to complete, may not give you well-performing scores without high-performing storage (i.e. SSD arrays).  Can you share the array details?

All of that aside, though, there seems to be a fundamental networking problem causing the "Guest Info failures".  If you SSH into your PrimeClient VM, are you able to ping the tile0 VMs (i.e. 'ping Standby0, 'ping DS3WebA0')?

View solution in original post

Reply
0 Kudos
9 Replies
dmorse
VMware Employee
VMware Employee
Jump to solution

Hi mathieugont

Sorry you're still having issues.  A few things I want to point out:

  • It looks like you have created a cluster for each ESXi host, and the first cluster, ClusterESXi001, has 1 host, but all of the tile 0 VMs on it.  This will not work for multiple reasons:
    • The VMmark3 benchmark is designed to vMotion, storage vMotion, etc. VMs from 1 host to another within the same cluster.
    • The benchmark is designed to only work with a single client ESXi host cluster (ClusterESXi005 in your case), and a single server ESXi host cluster (which you have broken out into ClusterESXi001,2,3,4).
    • Creating shared storage such as NFS, while it may allow the benchmark to complete, may not give you well-performing scores without high-performing storage (i.e. SSD arrays).  Can you share the array details?

All of that aside, though, there seems to be a fundamental networking problem causing the "Guest Info failures".  If you SSH into your PrimeClient VM, are you able to ping the tile0 VMs (i.e. 'ping Standby0, 'ping DS3WebA0')?

Reply
0 Kudos
mathieugont
Enthusiast
Enthusiast
Jump to solution

Hi David

I followed your recommendations grouping all my nodes within a single cluster (4 hardware servers grouped in ClusterESXi_BDW), except one (in ClusterESXi_Mgr) hosting the VMware vCenter Server Appliance (named photon), and a personal VM used to access vSphere client remotely (internal config). See the print screen below.

In ClusterESXi_BDW, all my VMs are hosted by the node 172.22.197.109, while my Client0 and my Prime Client are on the node 172.22.193.115.

From my Prime Client, I can ping and passwordless ssh all the VMs.

About a well-performing scores without high-performing storage (i.e. SSD arrays), the aim is to make it work first. Score and improvement will be treated in a second hand. For the moment, I need to run weathervane only, that is why in my property files, I set:

WorkloadList = AuctionLB,AuctionWebA,AuctionWebB,AuctionAppA,AuctionAppB,AuctionDB,AuctionMSQ,AuctionNoSQL

InfrastructureList =

But as you can see in the zip file attached to this message, comment "InfrastructureList =" to get the default does not solve the problem.

pastedImage_0.png 

Reply
0 Kudos
mathieugont
Enthusiast
Enthusiast
Jump to solution

Quick additional comment

  • My attached files are without infrastructure VMs and the Auction only on a single tile. Is WMmark is designed to work it? If not, should I deploy a second tile on another hardware server?
  • Message are finally different when I add infrastructure VMs. 
Reply
0 Kudos
dmorse
VMware Employee
VMware Employee
Jump to solution

Hi mathieugont

I checked with the technical lead for VMmark, he recommended rebooting your PrimeClient VM and double-checking that there aren't any duplicate IP addresses in your network.  It looks like something aborted the Guest Info collection on the PrimeClient -- these kinds of messages are very unusual:

20190531-18:07:34 Info  Process: localhost : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190531-18:07:34 Info  localhost : Get Guest Info did not run correctly

20190531-18:07:34 Info  Process: AuctionLB0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable does not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190531-18:07:34 Info  AuctionLB0 : Get Guest Info did not run correctly

20190531-18:07:34 Info  Process: Client0 : Get Guest Info failed to start/complete. Returned: RC = 1, STAFResult = This variable doesnot exist: VMMARK/AuctionNoSQL/SKIPRESTORE

20190531-18:07:34 Info  Client0 : Get Guest Info did not run correctly

Also -- if you want to just run Weathervane, that workload is an open-source project on github:

GitHub - vmware/weathervane: Virtual-Infrastructure and Cloud Performance Benchmark

Weathervane itself has a run harness that might be easier to use than trying to run VMmark without the other workloads, but it's up to you.

Reply
0 Kudos
mathieugont
Enthusiast
Enthusiast
Jump to solution

Thanks David

Did you saw the same message appeared whatever the VM:

Returned: RC = 1, STAFResult = This variabledoes not exist: VMMARK/AuctionNoSQL/SKIPRESTORE

RC=1 refers to API issues. Not a good sign 😉

But seems AuctionNoSQL is the issue.

Any comment on that?

Reply
0 Kudos
mathieugont
Enthusiast
Enthusiast
Jump to solution

David

I am trying to run weathervane out of VMmark like you advised to me.

I followed the procedures described in section 3 and 4 of the manual. I needed to tune the scripts, but it the end, it does not work with the following message:

Wed Jun 5 14:46:29 2019: Weathervane Version 1.0.21

Wed Jun 5 14:46:29 2019: Command-line parameters:

Wed Jun 5 14:46:29 2019: Run Configuration has 1 workloads.

Wed Jun 5 14:46:29 2019: Workload 1 has 1 workload-driver nodes

Wed Jun 5 14:46:29 2019: Workload 1 has 1 application instances.

Wed Jun 5 14:46:29 2019: Workload 1, Application Instance 1 configuration:

Wed Jun 5 14:46:29 2019:        1 ConfigurationManagers

Wed Jun 5 14:46:29 2019:        0 ElasticityServices

Wed Jun 5 14:46:29 2019:        1 CoordinationServers

Wed Jun 5 14:46:29 2019:        0 LbServers

Wed Jun 5 14:46:29 2019:        1 WebServers

Wed Jun 5 14:46:29 2019:        1 DbServers

Wed Jun 5 14:46:29 2019:        1 NosqlServers

Wed Jun 5 14:46:29 2019:        0 FileServers

Wed Jun 5 14:46:29 2019:        1 MsgServers

Wed Jun 5 14:46:29 2019:        1 AppServers

Wed Jun 5 14:46:29 2019:        maxDuration = 7200

Wed Jun 5 14:46:29 2019: Running Weathervane with Single-Run RunManager using Full-Run RunProcedure.

Wed Jun 5 14:46:29 2019: Single-Run RunManager starting run.

Wed Jun 5 14:46:30 2019: Stopping running services and cleaning up old log and stats files.

Wed Jun 5 14:46:48 2019: Preparing data for use in current run.

Wed Jun 5 14:46:48 2019: Configuring and starting data services for appInstance 1 of workload 1.

Wed Jun 5 14:47:40 2019: Backup is not available at the proper scale for 1000 users for appInstance 1 of workload 1. Loading data.

Wed Jun 5 14:48:29 2019: Workload 1, appInstance 1: Loading data for a maximum of 1000 users

Wed Jun 5 14:49:00 2019: Can't find dbloader pid for workload 1, appInstance 1

Wed Jun 5 14:49:29 2019: Could not properly load or prepare data for run 13.  Exiting.

Any idea of what is going wrong?

Reply
0 Kudos
RebeccaG
Expert
Expert
Jump to solution

Hi mathieugont​, I'm working on this and will get you an update tomorrow.

Reply
0 Kudos
mathieugont
Enthusiast
Enthusiast
Jump to solution

Thanks a lot

Take your time: Monday is off here 😉

Reply
0 Kudos
RebeccaG
Expert
Expert
Jump to solution

Hi, I noticed you are using a non-default name for VMmark3.properties. Could you try reverting your file VMmark3.tile0.weathervane-auction.properties back to the default name VMmark3.properties (and following the instructions in the User's Guide to support that under "Starting a VMmark Run Through The GUI"). Also please comment out the line DS3DB/DS3DBsize = 100.

Also, could you please zip the results file from your most recent run and attach it to this thread.


Thank you,

Rebecca

Reply
0 Kudos