VMmark3 ClusterMapping Failed

[patelch]

VMmark3 Benchmark Running, please wait...
Response
--------
{
  Job ID         : 1
  Start Date-Time: 20220623-13:26:18
  End Date-Time  : 20220623-13:29:17
  Status         : Terminated
  Result         : None
  Job Log Errors : []
  Testcase Totals: {
    Tests : 2
    Passes: 1
    Fails : 1
  }
}
==> /root/VMmark3/results/Results_20220623132618_Tile_1_Bench1/VMmark3-STAX_Job_1_User-Parsed.log <==
20220623-13:28:55 Info  VMmark3 : Checking Client Time Sync
20220623-13:28:57 Info  Service: Request vCServer Time Info : 10.253.96.178
20220623-13:28:58 Info  Info: All Clients and vCServer Times Within 60 Second Threshold
20220623-13:28:58 Info  Service: Request Cluster ListVMs Info : Dell_ISS_Cluster
20220623-13:28:58 Info  Service: Request Cluster CheckDrs Info : Dell_ISS_Cluster : Iteration 0
20220623-13:28:58 Info  SetupDir /root/VMmark3/results/Results_20220623132618_Tile_1_Bench1/setupfiles/
20220623-13:28:58 Info  Service: Request Cluster ListLUNs Info : Dell_ISS_Cluster
20220623-13:28:58 Info  Service: Request Cluster Mapping Info : Dell_ISS_Cluster (This May Take A While)
20220623-13:29:02 Info  VMs In Cluster : ['AuctionAppA0', 'AuctionNoSQL0', 'DS3DB0', 'DS3WebC0', 'ElasticWebA0', 'AuctionLB0', 'ElasticAppB0', 'AuctionAppB0', 'AuctionWebA0', 'DS3WebB0', 'DS3WebA0', 'AuctionDB0', 'ElasticDB0', 'ElasticLB0', 'ElasticWebB0', 'AuctionWebB0', 'ElasticAppA0', 'AuctionMSQ0', 'Standby0']
20220623-13:29:03 Info  VMmark3 ClusterMapping Failed to Run Correctly

See /root/VMmark3/results/Results_20220623132618_Tile_1_Bench1 for details

VMmark3 Benchmark Completed

I do not understand why my the ClusterMapping keeps failing. I am able to see the vms and hosts  on the cluster using the PowerCLI commands. All components are pingable and reachable via SSH.

[jamesz08]

Looking at the file VMmar3-ClusterMapping.log in Results_20220623134440_tile_1_benchV2\setupfiles

There seems to be an issue with the vmotion capability of the host dellissnode02.  Can you check all the hosts on your cluster to ensure that you can vmotion to each host?

[patelch]

I was able to deploy a Photon OS VM to the first node in the cluster and then vMotion compute and storage to all the nodes in the cluster and back to the original. I have attached the PowerCLI output of the VM tasks showing these actions. Please let me know if you require any further details. 

[jamesz08]

Can you remove the betaApiString from vmmark3.properties file and set the DebugLevel to 3 and rerun?

[patelch]

I have commented out the betaApiString and set the DebugLevel to 3. I have attached the results folder generated after the rerun.

[jamesz08]

Can you collect the output of the following on each host:

vim-cmd hostsvc/vmotion/netconfig_get

Also, can you verify that the time is consistent across all hosts (preferably ensure all hosts are set to the same ntp server).

[patelch]

I have confirmed that all the hosts are configured for the same NTP server. I have attached the vmotion config for each host. 

[jamesz08]

Do you have EVC configured for the cluster?

Can you run the attached PowerCLI against your cluster and post the output?

[patelch]

EVC is Enabled

           Mode: Intel® "Ice Lake" Generation

I have attached the output of the PowerCLI script you provided.

[jamesz08]

Ah sorry, there was a mistake in the script.  Can you run again with the attached:

[patelch]

The following is the output to the script:

Enter vCenter Server instance: dellissvcenter.wwtpoc.local

---- ---- ----
dellissvcenter.wwtpoc.local 443 VSPHERE.LOCAL\Administrator
Enter cluster name: Dell_ISS_Cluster
Do you want to perform vMotion network test? This will migrate specified VM across the hosts ensuring the network connectivity (y/n): y
Enter test VM name: Standby0
Testing VM vMotion compatibility between dellissnode01.wwtpoc.local and dellissnode02.wwtpoc.local
Testing VM vMotion compatibility between dellissnode02.wwtpoc.local and dellissnode03.wwtpoc.local
Testing VM vMotion compatibility between dellissnode03.wwtpoc.local and dellissnode04.wwtpoc.local
Testing VM vMotion compatibility between dellissnode04.wwtpoc.local and dellissnode01.wwtpoc.local

[jamesz08]

Hmm, it looks like it should work.  can you try running again with EVC disabled?

[patelch]

I disabled EVC and reran the script resulting in the same error.

[jamesz08]

You reran the benchmnark?  Can you try just setting DoClusterMapping = False in vmmark3.properties file and posting results?

[patelch]

I have attached the results file after changing the DoClusterMapping to false.

[jamesz08]

The current issue you have is that the Infrastructure opertations svmotion and xvmotion need a 2nd datastore to migrate the VMs to.  Currently you only have a single vsan datastore.  You will need to mount a 2nd datastore and set the following to point to this datastore in VMmark3.properties:

SVmotion/SVMotionLUNs = vsanDatastore

XVmotion/XVMotionLUNs = vsanDatastore

Alternatively you can disable these operations by setting the following:

InfrastructureList = vmotion,deploy

[patelch]

I disabled the svmotion and xvmotion operations and the run still failed. I have attached the results folder.

[jamesz08]

Deploy is failing setup due to missing template.  

You have the template set as:

Deploy/Templates = vmmark3.1.1-template-04022022-1

This should be the name of the template VM you used for provisioning.

Your provisioning source is 

ProvisioningSource = vmmark3.1.1-template-04202022-1

It looks like you have a typo for the deploy template.  

[patelch]

The benchmark run completed successfully for the tile. Thank you for all your help.