All Posts

Thank you Oporras for uploading your results files. These are the VMmark storage infrastructure operations VMs: (quoted from VMmark3.properties file) // SVmotion/Workloads = Standby // XVmotion/Workloads = DS3WebA Please make sure that these VMs for every tile are located on a different datastore from ones you specified in VMmark3.properties: SVmotion/SVMotionLUNs = datastore1,datastore1 XVmotion/XVMotionLUNs = datastore1,datastore1 You can actually use the same datastore for all infrastructure operations as shown above, but they can also all be different.   The error you are seeing is because the workload VM in question is present on the same datastore as specified in SVMotionLUNs or XVMotionLUNs. It needs to be on a different datastore. This is why VMmark 3 requires at least 2 shared datastores across the SUT cluster.   In file Results_20231029035855_tile0, you have  SVmotion/SVMotionLUNs = DS_CLIPTEST01,DS_CLIPTEST02   You stated:  DS_CLIPTEST01 >   Standby0    Standby0 is on DS_CLIPTEST01, that is the source of the problem. Please migrate Standby0 to a different datastore or specify a different datastore in SVmotion/SVMotionLUNs.   In Results_20231029031758, SVmotion/SVMotionLUNs = DS_AMSA_QA_DEV,DS_BASESWIN_QA_01 You stated: DS_AMSA_QA_DEV >   Standby0  Same problem.   Also, I did not do a comprehensive review of your setup but I wanted to mention a couple of things I noticed: 1. Make sure you do not run VMmark on a production environment. There were many other VMs on your VMmark cluster.  From the VMmark 3 User's Guide, "NOTE VMmark, and the benchmark hardware and software components (vCenter, ESXi, and the VMmark template, clients, workload virtual machines, systems under test, storage, etc.) must be used only in an isolated test environment. None of the benchmark hardware or software components should ever be connected to a production network or transitioned to a production environment." Among other things, when you run VMmark it will create terrible performance for any production applications and produce invalid VMmark 3 results. 2. You have specified the same cluster for both the VMmark 3 Client cluster and SUT cluster. These need to be separate clusters and physical machines. You can still run VMmark with this configuration, for example, for functional testing, but it will produce invalid performance results.   Thank you, Rebecca

Hi oporras, VMmark is typically used to run as many tiles as possible in a cluster and still achieve a passing score. To find a rough estimate of how many tiles your cluster will support, you can look at the VMmark results page. Try filtering results by the total number of cores you have in your cluster. You may also be able to find a published VMmark result which uses the same or similar processor to your cluster, and extrapolate the number of tiles. You are also thinking about the memory and storage requirements for VMmark, that is good. Check the "System Requirements" on the VMmark website https://www.vmware.com/products/vmmark.html as it lists the amount of memory and storage needed for each tile. After you provision VMmark 3 with the tiles you calculated, see whether the run passed or failed (see the VMmark User's Guide for details).  If it passed, you need to provision more tiles so you can find the largest number of tiles that passes. If it failed you need to run VMmark 3 with one less tile. Continue until you have a passing run.

Hi oporras, VMmark 3 is typically used to run as many tiles as possible in a cluster and still achieve a passing score. To find a rough estimate of how many tiles your cluster will support, you can look at the VMmark results page. Try filtering results by the total number of cores you have in your cluster. You may also be able to find a published VMmark result which uses the same or similar processor to your cluster, and extrapolate the number of tiles. You are also thinking about the memory and storage requirements for VMmark, that is good. Check the "System Requirements" on the VMmark website https://www.vmware.com/products/vmmark.html as it lists the amount of memory and storage needed for each tile. After you provision VMmark 3 with the tiles you calculated, see whether the run passed or failed (see the VMmark User's Guide for details).  If it passed, you need to provision more tiles so you can find the largest number of tiles that passes. If it failed you need to run VMmark 3 with one less tile. Continue until you have a passing run.

Hi Wen, this is great. I see you resolved your earlier problems and are running 10 tiles with no setup issues. Number_of_Compliance_Issues (identified by '*' or '+') : 1 Issues Found : Tile4-weathervaneauction-p1 Median_Phase : p1 This is a Quality of Service (QoS) compliance issue. It means that the VMmark run failed to meet quality of service (i.e. Application Latency) requirements due to resource contention. This is normal when you are running a large number of tiles as eventually the QoS will fail due to storage contention, CPU contention, or other resource constraints. The failing tile, tile 4 is just barely failing. A passing score is less than 1. In VMmark3-graphs.html, it shows WeathervaneAuction has two spikes in QoS during the middle of the run. This is why the QoS failed. If you can resolve the cause of this poor performance, you might be able to pass a VMmark run at 10 tiles or more. You could also run VMmark again, and might be able to pass if you get lucky (slightly better performance from your environment). Thank you, Rebecca

Hi RebeccaG I have a issue in the Score.txt . Number_of_Compliance_Issues (identified by '*' or '+') : 1 Issues Found : Tile4-weathervaneauction-p1 Median_Phase : p1 I have uploaded the zip log on the ftp, the the directory name is 'Wen',and the filenames are Results_20231023072053_10tiles_2.zip.00*. Please use 7-zip to decompress the six packages together. Thank You Wen

Hi Wen, It should be ok if those hosts are in a different datacenter with same vCenter Server. For the new error, please zip your VMmark results file and if small enough, attach to this forum thread. If the zip file is too large please put it on ftpsite like last time. Also please try to manually vmotion or storage vmotion your VMmark VMs as it is failing in your environment. Also please check why your vMotion LUN shows error from my last post, it is the column 'uncommitted' which shows the error. Your file VC-ListLUNs.txt also shows an error with the vmotion LUN: Name : Free Space : Capacity : Uncommitted : Type : Multiple Host Access : Maintenance Mode VMotion : 7819GB : 7821GB : ERROR : VMFS : true : normal Second, all infrastructure operations (vmotion, xvmotion, svmotion, deploy) .wrf files show the response "FAIL". This error from vCenter Server means the operation cannot complete. To troubleshoot this, replicate the activity manually: Using the vCenter UI, manually vmotion, storage vmotion or combined vmotion and storage vmotion a VMmark VM from one host to the other host in the SUT cluster. Does it complete successfully? If not, watch the error that vCenter Server returns. You will need to fix this before you can run VMmark.

Hi RebeccaG, Here is another problem as below.Why the test runs too short. 20231013-02:00:50 Info Error: tile 1 DS3WebA run too short, elapsed time = 50,expected time = 189 20231013-02:00:50 Info Error: tile 1 AuctionLB run too short, elapsed time = 50, expected time = 11220 20231013-02:01:50 Info Error: tile 2 AuctionLB run too short, elapsed time = 50, expected time = 11160 20231013-02:01:51 Info Error: tile 2 DS3WebA run too short, elapsed time = 51,expected time = 188 20231013-02:02:50 Info Error: tile 3 DS3WebA run too short, elapsed time = 50,expected time = 187 20231013-02:02:50 Info Error: tile 3 AuctionLB run too short, elapsed time = 50, expected time = 11100 20231013-02:03:50 Info Error: tile 4 DS3WebA run too short, elapsed time = 50,expected time = 186 20231013-02:03:50 Info Error: tile 4 AuctionLB run too short, elapsed time = 50, expected time = 11040 20231013-02:04:50 Info Error: tile 5 AuctionLB run too short, elapsed time = 50, expected time = 10980 20231013-02:04:51 Info Error: tile 5 DS3WebA run too short, elapsed time = 51,expected time = 185 20231013-02:05:50 Info Error: tile 6 AuctionLB run too short, elapsed time = 50, expected time = 10920 20231013-02:05:51 Info Error: tile 6 DS3WebA run too short, elapsed time = 51,expected time = 184 20231013-02:06:50 Info Error: tile 7 DS3WebA run too short, elapsed time = 50,expected time = 183 20231013-02:06:50 Info Error: tile 7 AuctionLB run too short, elapsed time = 50, expected time = 10860 20231013-02:07:50 Info Error: tile 8 AuctionLB run too short, elapsed time = 50, expected time = 10800 20231013-02:07:51 Info Error: tile 8 DS3WebA run too short, elapsed time = 51,expected time = 182 20231013-02:08:50 Info Error: tile 9 DS3WebA run too short, elapsed time = 50,expected time = 181 20231013-02:08:50 Info Error: tile 9 AuctionLB run too short, elapsed time = 50, expected time = 10740 20231013-02:09:50 Info Error: tile 10 AuctionLB run too short, elapsed time =50, expected time = 10680 20231013-02:09:51 Info Error: tile 10 DS3WebA run too short, elapsed time = 51, expected time = 180 20231013-05:25:06 Info Tile 0 : Prime Client Time 1697174705 UTC : Client0 Time 1697174702 UTC :: TimeErrors : 0 :: Clock Skew 3 20231013-05:25:08 Info Tile 1 : Prime Client Time 1697174707 UTC : Client1 Time 1697174705 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:10 Info Tile 2 : Prime Client Time 1697174709 UTC : Client2 Time 1697174707 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:13 Info Tile 3 : Prime Client Time 1697174711 UTC : Client3 Time 1697174709 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:15 Info Tile 4 : Prime Client Time 1697174714 UTC : Client4 Time 1697174711 UTC :: TimeErrors : 0 :: Clock Skew 3 20231013-05:25:17 Info Tile 5 : Prime Client Time 1697174716 UTC : Client5 Time 1697174714 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:19 Info Tile 6 : Prime Client Time 1697174718 UTC : Client6 Time 1697174716 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:22 Info Tile 7 : Prime Client Time 1697174720 UTC : Client7 Time 1697174718 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:24 Info Tile 8 : Prime Client Time 1697174723 UTC : Client8 Time 1697174721 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:26 Info Tile 9 : Prime Client Time 1697174725 UTC : Client9 Time 1697174723 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:28 Info Tile 10 : Prime Client Time 1697174727 UTC : Client10 Time 1697174725 UTC :: TimeErrors : 0 :: Clock Skew 2 20231013-05:25:32 Info Error: Tile 2 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 3 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 1 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 6 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 7 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 4 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 8 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 5 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 9 Weathervane results number not correctlyidentified 20231013-05:25:32 Info Error: Tile 10 Weathervane results number not correctlyidentified Thanks, Wen

Hi RebeccaG The vCenter has two datacenter for two tests. hosts 198.168.55.112 and .113,are the first test suts, 198.168.55.68 and .69,are the first suts' clients. .111 and 110 are the second test suts and .67 and .66 are the suts' cliemts. The two tests use different storage but the same vCenter. If it affects the test. Thank you , Wen

Hi Wen, yes I successfully unzipped your file using 7-zip, thank you. There are errors causing Weathervane to not run and all infrastructure operations (vmotion, svmotion, xvmotion, deploy) are failing. I think you have some issues with your shared storage and possibly vCenter Server. The log VC-CollectEvents.csv shows many storage connectivity errors on hosts 198.168.55.112 and .113, 198.168.55.68 and .69, and these hosts are in SUTCluster and ClientCluster respectively. However, VC-ListHosts.txt shows only .111 and 110 in your SUTCluster and .67 and .66 in ClientCluster. Were these other hosts maybe in maintenance mode? Either way these storage errors could be affecting your vmmark run.  I would recommend you remove any unused hosts / hosts with errors from your VMmark clusters. Your file VC-ListLUNs.txt also shows an error with the vmotion LUN: Name : Free Space : Capacity : Uncommitted : Type : Multiple Host Access : Maintenance Mode VMotion : 7819GB : 7821GB : ERROR : VMFS : true : normal Second, all infrastructure operations (vmotion, xvmotion, svmotion, deploy) .wrf files show the response "FAIL". This error from vCenter Server means the operation cannot complete. To troubleshoot this, replicate the activity manually: Using the vCenter UI, manually vmotion, storage vmotion or combined vmotion and storage vmotion a VMmark VM from one host to the other host in the SUT cluster. Does it complete successfully? If not, watch the error that vCenter Server returns. You will need to fix this before you can run VMmark. Thank you and please let me know if you have any further questions, Rebecca  

Hi RebeccaG， Because I failed to upload the package “Results_20231011065120_test1_2.zip”directly for Authentication fails,so I compressed the package again by volume. Can you use 7-zip to decompress the three packages together? Thanks, Wen

Hi Wen, I received three files from you:  Results_20231011065120_test1_2.zip.001,Results_20231011065120_test1_2.zip.002,Results_20231011065120_test1_2.zip.003. I'm having difficulty extracting the files. Can you please use a command like this on the Prime Client to zip your VMmark results files?  zip -r Results_20190326174141.zip Results_20190326174141 Thank you, Rebecca

HI RebeccaG， The vmmark test failed in the last test ,and I upload the log on the ftp. the the directory name is 'Wen',and the filenames are Results_20231011065120_test1_2.zip.001,Results_20231011065120_test1_2.zip.002,Results_20231011065120_test1_2.zip.003. I can upload them together while it always fails for Authentication Thans， Wen