jiangjing123455
Contributor
Contributor

Weathervane Exceptions

Jump to solution

Hi,

When I started the vmmark test with 4 tiles,the result is "Run_Is_NOT_Compliant".

Warnings Messages::
  p0 : WeathervaneAuction0 Exceptions : 2
  p0 : WeathervaneElastic0 Exceptions : 1
  p1 : WeathervaneAuction0 Exceptions : 4
  p2 : WeathervaneAuction0 Exceptions : 1
  p2 : WeathervaneElastic0 Exceptions : 119
  rampdown : WeathervaneAuction0 Exceptions : 1
  rampdown : WeathervaneElastic0 Exceptions : 120
  p0 : WeathervaneElastic1 Exceptions : 2
  p1 : WeathervaneAuction1 Exceptions : 1
  rampdown : WeathervaneElastic1 Exceptions : 95
  p0 : WeathervaneAuction2 Exceptions : 1
  p0 : WeathervaneElastic2 Exceptions : 1
  p1 : WeathervaneElastic2 Exceptions : 143
  p2 : WeathervaneAuction2 Exceptions : 1
  p2 : WeathervaneElastic2 Exceptions : 173
  rampdown : WeathervaneElastic2 Exceptions : 130
  p0 : WeathervaneAuction3 Exceptions : 3
  p0 : WeathervaneElastic3 Exceptions : 2
  p1 : WeathervaneElastic3 Exceptions : 155
  p2 : WeathervaneElastic3 Exceptions : 183
  rampdown : WeathervaneAuction3 Exceptions : 4
  rampdown : WeathervaneElastic3 Exceptions : 128
 
Summary ::
Run_Is_NOT_Compliant
Turbo_Setting : 0
Number_of_Workloads_Missing : 0
Number_of_Compliance_Issues (identified by '*' or '+') : 1
Issues Found : 
    Tile3-weathervaneelastic-p0
Median_Phase : p2
 
 I have two VMware clusters for the SUT and the client systems and an SAN storage system with NVMEs providing the shared iSCSI datastores for infrastructure operations. All systems are connected via  10Gbps ethernet switches.
 
I looked in the wrf files from weathervane, and I see the following sampling of exceptions:
 
13:06:14.481 [pool-3-thread-36] WARN c.v.w.w.common.core.Operation - Operation:run restarting userId = 786, operation = GetNextBid, behavior UUID ac322b20-5065-4b90-a786-8277c875b419 Failure Reason = io.netty.handler.timeout.ReadTimeoutException
13:06:15.586 [pool-3-thread-92] WARN c.v.w.w.common.core.Operation - Operation:run restarting userId = 278, operation = GetNextBid, behavior UUID 25cbb910-0112-4722-9d49-fdc1ae1f8c6d Failure Reason = io.netty.handler.timeout.ReadTimeoutException
 
11:50:42.764 [pool-3-thread-15] WARN c.v.w.w.common.core.Operation - Operation:run Execution Failed for GetNextBid for behavior UUID 98cc3664-6c6a-4dec-9f2e-64f426f4d9d8 Failure Reason = com.vmware.weathervane.workloadDriver.common.exceptions.OperationFailedException: Incomplete response received when retrieving current bid for auction 30171
11:50:42.766 [pool-3-thread-15] WARN c.v.w.w.common.core.Operation - Operation:run restarting userId = 2371, operation = GetNextBid, behavior UUID 98cc3664-6c6a-4dec-9f2e-64f426f4d9d8 Failure Reason = com.vmware.weathervane.workloadDriver.common.exceptions.OperationFailedException: Incomplete response received when retrieving current bid for auction 30171
 
13:56:21.947 [pool-3-thread-58] WARN c.v.w.w.common.core.Operation - Operation:run Execution Failed for GetNextBid for behavior UUID 71f3adb0-3690-438c-b737-3363419eba37 Failure Reason = java.lang.NullPointerException
13:56:21.947 [pool-3-thread-58] WARN c.v.w.w.common.core.Operation - Operation:run restarting userId = 4626, operation = GetNextBid, behavior UUID 71f3adb0-3690-438c-b737-3363419eba37 Failure Reason = java.lang.NullPointerException
 
Can you please advise what else could be  causing these issues? Thanks.
0 Kudos
1 Solution

Accepted Solutions
dmorse
VMware Employee
VMware Employee

Hi @jiangjing123455 , I am not sure why only the first test passes.

If you shut down all 5 tiles using this command from the prime client (make sure you're in the /root/VMmark3 directory first):

java -jar tools/VMmark3Service.jar -c VMmark3.properties -m tilePower -tiles 0

Then start all 5 tiles using this command:
java -jar tools/VMmark3Service.jar -c VMmark3.properties -m tilePower -tiles 5

And then try another 5-tile run, does that pass?

View solution in original post

0 Kudos
3 Replies
dmorse
VMware Employee
VMware Employee

Hi @jiangjing123455 , I am glad to see you were able to scale up to this level.

  • 1 compliance issue is very low; have you tried more than one run?  It's quite possible that rerunning may produce a compliant run.
  • You mentioned you have SAN storage for the workload VMs; is it all-flash?
  • You may want to consider grouping the VMs onto separate LUNs, since some are more I/O intensive, and others are more latency sensitive.

It sounds like the benchmark is running correctly, so now it is just a matter of performance tuning your environment.  Reviewing the published VMmark results (https://www.vmware.com/products/vmmark/results3x.0.html), specifically the "Storage Notes" section of the PDFs that use SAN and vSAN storage, should help here.

jiangjing123455
Contributor
Contributor

Hi,dmorse

I started the vmmark test with 5 tiles by taking your suggestion,but I only passed the test with 5 tiles for first run.

When I started the wmmark test with 5 tiles for the first time,the result was "Run_Is_Compliant" .

Warnings Messages::
  p0 : WeathervaneAuction0 Exceptions : 4
  p0 : WeathervaneElastic0 Exceptions : 189
  p1 : WeathervaneAuction0 Exceptions : 1
  p1 : WeathervaneElastic0 Exceptions : 345
  p2 : WeathervaneElastic0 Exceptions : 365
  rampdown : WeathervaneElastic0 Exceptions : 323
  p0 : WeathervaneAuction1 Exceptions : 2
  p0 : WeathervaneElastic1 Exceptions : 103
  p1 : WeathervaneAuction1 Exceptions : 1
  p1 : WeathervaneElastic1 Exceptions : 280
  p2 : WeathervaneAuction1 Exceptions : 1
  p2 : WeathervaneElastic1 Exceptions : 602
  rampdown : WeathervaneElastic1 Exceptions : 368
  p0 : WeathervaneAuction2 Exceptions : 1
  p0 : WeathervaneElastic2 Exceptions : 82
  p1 : WeathervaneElastic2 Exceptions : 214
  p2 : WeathervaneElastic2 Exceptions : 290
  rampdown : WeathervaneElastic2 Exceptions : 336
  p0 : WeathervaneElastic3 Exceptions : 117
  p1 : WeathervaneAuction3 Exceptions : 2
  p1 : WeathervaneElastic3 Exceptions : 185
  p2 : WeathervaneAuction3 Exceptions : 1
  p2 : WeathervaneElastic3 Exceptions : 241
  rampdown : WeathervaneElastic3 Exceptions : 386
  p0 : WeathervaneAuction4 Exceptions : 9
  p0 : WeathervaneElastic4 Exceptions : 65
  p1 : WeathervaneAuction4 Exceptions : 1
  p1 : WeathervaneElastic4 Exceptions : 195
  p2 : WeathervaneElastic4 Exceptions : 170
  rampdown : WeathervaneElastic4 Exceptions : 160
 
Summary ::
Run_Is_Compliant
Median_Phase : p2
 
But when I started the wmmark test with 5 tiles for the second time,the result was "Run_Is_Not_Compliant" and the number of  'warnings_messages' became more and more.
 
Warnings Messages::
  p0 : WeathervaneAuction0 Exceptions : 1
  p0 : WeathervaneElastic0 Exceptions : 268
  p1 : WeathervaneElastic0 Exceptions : 345
  p2 : WeathervaneElastic0 Exceptions : 333
  rampdown : WeathervaneElastic0 Exceptions : 381
  p0 : WeathervaneAuction1 Exceptions : 2
  p0 : WeathervaneElastic1 Exceptions : 102
  p1 : WeathervaneElastic1 Exceptions : 261
  p2 : WeathervaneAuction1 Exceptions : 1
  p2 : WeathervaneElastic1 Exceptions : 513
  rampdown : WeathervaneElastic1 Exceptions : 357
  p0 : WeathervaneAuction2 Exceptions : 1
  p0 : WeathervaneElastic2 Exceptions : 20
  p1 : WeathervaneAuction2 Exceptions : 1
  p1 : WeathervaneElastic2 Exceptions : 215
  p2 : WeathervaneAuction2 Exceptions : 1
  p2 : WeathervaneElastic2 Exceptions : 246
  rampdown : WeathervaneElastic2 Exceptions : 282
  p0 : WeathervaneAuction3 Exceptions : 2
  p0 : WeathervaneElastic3 Exceptions : 68
  p1 : WeathervaneElastic3 Exceptions : 211
  p2 : WeathervaneElastic3 Exceptions : 314
  rampdown : WeathervaneElastic3 Exceptions : 327
  p0 : WeathervaneAuction4 Exceptions : 31033
  p0 : WeathervaneElastic4 Exceptions : 70
  p1 : WeathervaneAuction4 Exceptions : 29799
  p1 : WeathervaneElastic4 Exceptions : 212
  p2 : WeathervaneAuction4 Exceptions : 29433
  p2 : WeathervaneElastic4 Exceptions : 292
  rampdown : WeathervaneAuction4 Exceptions : 19909
  rampdown : WeathervaneElastic4 Exceptions : 384
 
Summary ::
Run_Is_NOT_Compliant
Turbo_Setting : 0
Number_of_Workloads_Missing : 0
Number_of_Compliance_Issues (identified by '*' or '+') : 2
Issues Found : 
    Tile2-weathervaneelastic-p0
    Tile4-weathervaneelastic-p0
Median_Phase : p2

 

This situation also appeared in the third test and the fourth test.

Can you please advise what else could be  causing these issues? Thanks.

0 Kudos
dmorse
VMware Employee
VMware Employee

Hi @jiangjing123455 , I am not sure why only the first test passes.

If you shut down all 5 tiles using this command from the prime client (make sure you're in the /root/VMmark3 directory first):

java -jar tools/VMmark3Service.jar -c VMmark3.properties -m tilePower -tiles 0

Then start all 5 tiles using this command:
java -jar tools/VMmark3Service.jar -c VMmark3.properties -m tilePower -tiles 5

And then try another 5-tile run, does that pass?

View solution in original post

0 Kudos