Hello all!
I've been testing vSphere 8 with a couple new machines and more specifically the VCSA install using vSAN ESA with one NVMe drive. The install worked great on two different machines, but now there are issues.
Short story:
In Install Stage 1, Step 6 Select datastore - There is an error "Upload HCL DB failed: Failed to replace old HCL DB with new one." This error prevents vSAN ESA from being selected.
Long story:
On the first machine, everything worked perfectly. VCSA installed with no issues, on Step 6 the vSAN ESA option was available and vCenter was installed with a single node vSAN cluster and that node had a single NVMe drive. Since this was just the first test, I didn't leave this running very long.
On the second machine, everything worked exactly like the first. This one I let run for a couple days. Then for some reason, I wanted to reboot the host, so I shut down the vCenter VM, put the host into Maintenance Mode and rebooted it. When it came back up, there was no datastore and the vCenter VM wasn't in the inventory. Weird, but ok maybe some issue with rebooting a single node vSAN cluster node, I'll just rebuild again.
Now on both machines... When I run the VCSA installer, the introduction screen has an alert "The External Platform Services Controller deployment has been deprecated". I didn't notice this before and I'm pretty sure it wasn't there. Then on Step 6 as I mentioned above, there is the error about replacing the HCL DB. At the bottom of the screen is an alert with the same test as the error above, and an option to Retry. Screenshots for both attached.
What I've tried:
The Retry option I mentioned above will try once, and then it shows an option of uploading a new HCL DB. On the Hardware Compatibility Matrix site, if vSAN is selected, then there is an option for downloading an all.json file which can be uploaded in the VCSA installer. But this didn't work.
Thinking maybe a copy of this HCL DB might be stored on the ESXi host, I rebuilt the host from scratch and removed all the partitions from the NVMe drive. Same issue. Then I thought this HCL DB must be local to the machine that ran the VCSA installer, so I ran it from a different VM, same issue.
There is a PowerCLI command that can manually update the HCL DB, but it looks like it is intended to have vCenter as the target. But, I tried running it on the ESXi host anyway and got an error saying it cannot be run the host. Good try.
Then I ran a packet capture on this vlan during the install. Obviously most of the traffic is between the machine running the installer and the host, but the machine running the installer does talk out to a couple IPs. So I tried running the install first with all firewall outbound ports allowed, then all outbound ports blocked. No difference either way.
The install does put some temp files out in the user profile AppData\Local\Temp folder, so I tried deleting those. No difference. I haven't had the energy to try searching through the registry yet, but I don't really think that would change anything since I've tried the install from a different VM.
Any ideas or other things to try?
