Cluster after move and repair of VSAN has error of "Update ESXI Configuration" for update ESXI Configuration. All three hosts note :
Last Update by VC
Different VC (60f584a0-1d04-3c42-154b-a0423f377a7e)
Running wizard for vCenter to take over ownership of vSAN completes but never clears error.
My assumption is the UUID "60f584a0-1d04-3c42-154b-a0423f377a7e" was the UUID of the temp vCenter I used to repair the cluster. But not sure how to get it to let go and revert under this vCenter. Typical vCenter lack of detail logging to further root cause things 😛
<poke on this thread>
Been working on other fires.. back to this topic:
Back working on this project.
Run update : event task list shows success but hosts still listed as needing to be set to authoritative.
tail /var/ syslog.log
2021-09-08T20:33:21Z backup.sh[2381243]: Creating ConfigStore Backup
2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed successfully
2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed with rc = 101
2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed successfully
2021-09-08T20:33:21Z configStoreBackup: ConfigStore backup completed with rc = 101
2021-09-08T20:33:22.575Z ConfigStore[2381360]: Log for ConfigStore version=1.0 build=build-17867351 option=Release
2021-09-08T20:33:22.575Z ConfigStore[2381360]: Could not expand environment variable HOME.
2021-09-08T20:33:22.575Z ConfigStore[2381360]: Could not expand environment variable HOME.
2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "/usr/lib/vmware/config": No such file or directory.
2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "~/.vmware/config": No such file or directory.
2021-09-08T20:33:22.575Z ConfigStore[2381360]: DictionaryLoad: Cannot open file "~/.vmware/preferences": No such file or directory.
2021-09-08T20:33:22.575Z ConfigStore[2381360]: Switching to VMware syslog extensions
2021-09-08T20:33:23Z backup.sh[2381243]: Locking esx.conf
2021-09-08T20:33:23Z backup.sh[2381243]: Creating archive
2021-09-08T20:33:23Z backup.sh[2381243]: Unlocking esx.conf
2021-09-08T20:33:24Z backup.sh[2381243]: Using key ID 5c4c03bf-c118-48e3-a03c-1d34080191a3 to encrypt
[root@thor:~] tail -f /var/log/vmkernel.log
2021-09-08T20:59:00.173Z cpu25:2098895 opID=1afa4246)World: 11986: VC opID 11232485-W773-069e maps to vmkernel opID 1afa4246
2021-09-08T20:59:00.173Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomRepairDelay" = 60, Old Value: 60, (Status: 0x0)
2021-09-08T20:59:00.189Z cpu25:2098895 opID=1afa4246)Config: 716: "DOMOwnerForceWarmCache" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.204Z cpu25:2098895 opID=1afa4246)Config: 716: "SwapThickProvisionDisabled" = 1, Old Value: 1, (Status: 0x0)
2021-09-08T20:59:00.212Z cpu25:2098895 opID=1afa4246)Config: 716: "goto11" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.213Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomBgProRebalanceEnabled" = 1, Old Value: 1, (Status: 0x0)
2021-09-08T20:59:00.220Z cpu25:2098895 opID=1afa4246)Config: 716: "ClomBgProRebalanceThreshold" = 30, Old Value: 30, (Status: 0x0)
2021-09-08T20:59:00.229Z cpu25:2098895 opID=1afa4246)Config: 716: "HostFailureThresholdState" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.233Z cpu25:2098895 opID=1afa4246)Config: 716: "InternalOpThresholdState" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.239Z cpu25:2098895 opID=1afa4246)RDT: RDTVSISetEnableRdma:2519: Rdma already disabled. Nothing to do.
2021-09-08T20:59:00.242Z cpu25:2098895 opID=1afa4246)Config: 716: "DedupScope" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.250Z cpu25:2098895 opID=1afa4246)Config: 716: "GuestUnmap" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.253Z cpu25:2098895 opID=1afa4246)Config: 716: "DomCompResyncThrottle" = 0, Old Value: 0, (Status: 0x0)
2021-09-08T20:59:00.565Z cpu26:2098902 opID=a511cdc6)World: 11986: VC opID 112324a5-06a5 maps to vmkernel opID a511cdc6
2021-09-08T20:59:00.565Z cpu26:2098902 opID=a511cdc6)RDT: RDTVSIGetSubClusterSecCfgMode:4774: Current security mode 0, state 0
2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
2021-09-08T20:59:07.272Z cpu23:2097247) min,KB max,KB minLimit,KB eMin,KB rMinPeak,KB name
2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
2021-09-08T20:59:07.272Z cpu23:2097247) 204800 204800 -1 204800 204800 host/vim/vmvisor/config-file-tracker
2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 1092 72312 python.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 132 132 uwWorldStore.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 worldGroup.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 0 70692 uw.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 vsiHeap.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 264 792 pt.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 288 288 cartelheap.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 0 0 uwshmempt.2383871
2021-09-08T20:59:07.272Z cpu23:2097247) 0 -1 -1 136 136 uwAsyncRemapHeap.2383871
2021-09-08T20:59:07.272Z cpu23:2097247)------------ ------------ ------------ ------------ ------------ ------------------------------
2021-09-08T20:59:09.018Z cpu14:2098904 opID=aaad51e0)World: 11986: VC opID 112324fa-06b4 maps to vmkernel opID aaad51e0
2021-09-08T20:59:09.018Z cpu14:2098904 opID=aaad51e0)RDT: RDTVSIGetSubClusterSecCfgMode:4774: Current security mode 0, state 0
<<< not looking like much >>>
Those are the outputs from log files from one server while I run vCenter upgrade
< Update>
I put another new 1TB SSD into the server to see if it was something with the disk
[root@odin:~] esxcfg-mpath -ld t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
sata.vmhba0-sata.0:2-t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
Runtime Name: vmhba0:C0:T2:L0
Device: t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
Device Display Name: Local ATA Disk (t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________)
Adapter: vmhba0 Channel: 0 Target: 2 LUN: 0
Adapter Identifier: sata.vmhba0
Target Identifier: sata.0:2
Plugin: HPP
State: active
Transport: sata
[root@odin:~] dd if=/dev/zero of=/vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________
dd: can't open '/vmfs/devices/disks/t10.ATA_____WDC_WDS100T2B0B2D00YS70__________________183533804564________': Function not implemented