I've run into a situation where after installing ESXi, I put in my IP information and then reboot the server. After the host comes back online all settings including the password are gone. Has anyone run into this before? I'm looking at hostd.log for help, any other logs that might help?
Is the host installed on SD-card? You might want to check if the local storage where ESXi is installed is healthy. Looks like the settings are stored in RAM and are not being written to disk.
Here's what I get:
[root@~] /sbin/backup.sh 1
Saving current state in /bootbank
Time: 14:03:58 Date: 08/24/2020 UTC
[root@~] ls -l /bootbank/state.tgz
-rw-r--r-- 1 root root 67039 Aug 24 14:03 /bootbank/state.tgz
[root@~] cat /var/spool/cron/crontabs/root
#min hour day mon dow command
1 1 * * * /sbin/tmpwatch.py
1 * * * * /sbin/auto-backup.sh
0 * * * * /usr/lib/vmware/vmksummary/log-heartbeat.py
*/5 * * * * /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
00 1 * * * localcli storage core device purge
*/10 * * * * /bin/crx-cli gc
This looks ok to me.
What you may check is if
/etc/vmware/esx.conf gets updated after making any changes, and whether this file (with its current state) is included in the state.tgz after a backup.
If the changes exist, reboot the host, then check the esx.conf again.
So it turns out, after chatting with support that the bootbank is getting mounted to /tmp, which makes sense that settings aren't persisting. We believe it's a BIOS FW related issue since the filesystem is getting mounted to /tmp prior to the boot media's module loading.
lrwxrwxrwx 1 root root 22 Aug 24 17:09 bootbank -> /tmp/_bootbank9vftlphl