VMware Cloud Community
fermo4learn
Contributor
Contributor

[ESXi] Orphaned/Unregistered VM returns after reboot

Could someone please assist me with this annoying issue?

Thanks!

System Info: 

6.7.0 Update 1 (Build 10302608)

Installed on a 8GB Flashdrive

2 SSD Datastores

Steps taken:

     1. Through the vSphere 6.7 web client, I've selected all VMs

     2. Click on "Actions", and click "Delete"

     3. VMs were removed

     4. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

     5. Selected each VM one at a time and clicked "Actions", "Unregistered" (Done this for each VM)

     6. After rebooting the ESXi Host, VM's Orphaned / Unregistered object returns with the same corresponding #

     7. Reformatted both Datastore's SSD Drives with low level format but this does not affect Orphaned / Unregistered objects

     8. Using SSH, used command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.

     9. VMs instantly get removed from the web client

     9. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

Solution:

Annoying but doesn't seem to take up any storage. I've reinstalled ESXi on another USB but in future cases I would like to know if there's another solution

Questions:

     1. What causes this issue?

     2. What can I do to prevent running into this issue in the future?

     3. What can I do so that they don't reappear after rebooting the ESXi host?

Screenshot:

pastedImage_22.png

Reply
0 Kudos
5 Replies
sjesse
Leadership
Leadership

Do you also have vcenter, I see your using the host client, but if you have vcenter the vms in that inventory and it may be readding it.

Reply
0 Kudos
continuum
Immortal
Immortal

I assume this is an issue with the state.tgz file.
It would be useful to try to create a config backup manually and then inspect the actual files.


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
fermo4learn
Contributor
Contributor

sjesse​ I don't have vCenter setup however, I did try setting up  the vCenter Server Appliance (vCSA) as an OVF import. I don't think this is related to this issue though.

continuum

I've looked into the "state.tgz" file that you mentioned that I found this KB:

Locally restoring an ESXi configuration from state.tgz backup (2043048)

  1. I enabled SSH on the ESXi Host
  2. WinSCP into the host
  3. At the root directory, found the soft-link that directs to  /bootbank/state.tgz -> /tmp/state.tgz
  4. I've made a backup of the "state.tgz" file and deleted it
  5. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding # and the "state.tgz" is recreated.
  6. After looking into the "state.tgz" compressed file I found a file that contained a list of my previous VMs - state.tgz\state\state\local\local\etc\vmware\hostd\vmInventory.xml
  7. I found the same file in my ESXi host under - ./etc/vmware/hostd/
  8. I've removed the entries but upon restarting the ESXi host the VM's return

Question:

1.  Is the "state.tgz" file basically a way "to restore the locally saved ESXi configuration" if I were to reinstall the ESXi host onto another flash drive?

Reply
0 Kudos
wila
Immortal
Immortal

Hi,

How long do you wait between deleting/unregistering those VMs and rebooting the host?

I'm asking this as with vSphere/ESXi you are basically running from a RAM disk. Something that most of the times is a very good thing, other times however it can bite you, especially if you do not realize that this is the case.

Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.

You can force this by running the following command manually.

/sbin/auto-backup.sh

Hope this helps,

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
fermo4learn
Contributor
Contributor

Hi wila

I've attempted to troubleshooting with taking information you provided in consideration as well as allowing enough time for the ESXi host to self update if it's already set to be "auto backup" by itself. These steps have been repeated over a week now.

Update:

     1. Using SSH, used the command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.

     2. Soft rebooted the ESXi host through the web client

     3. After rebooting the ESXi host, all VM's Orphaned / Unregistered object return with a corresponding #

     4. Also, a previously restored VM was lost and was not listed as a new detected Orphaned / Unregistered VM

     5. Luckily I was able to restored the VM by registering and finding the location path of it's .VMDK files

     6. Using SSH, used the command   /sbin/auto-backup.sh and allowed enough time for the ESXi to

Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.

     7. After using SSH again to unregister VM's and soft rebooting the ESXi host, it reverts back to the corrupted state ( 3. Orphaned / Unregistered )

CMD log: /sbin/auto-backup.sh

[root@bvzesxi:~] /sbin/auto-backup.sh

         --- /etc/vmware/hostd/vmAutoStart.xml

         +++ /tmp/auto-backup.2100205//etc/vmware/hostd/vmAutoStart.xml

         @@ -1,20 +1,7 @@

          <ConfigRoot>

            <AutoStartOrder>

         -    <_length>1</_length>

         +    <_length>0</_length>

              <_type>vim.host.AutoStartManager.AutoPowerInfo[]</_type>

         -    <e id="0">

         -      <_type>vim.host.AutoStartManager.AutoPowerInfo</_type>

         -      <key>

         -        <_type>vim.VirtualMachine</_type>

         -        <moid>22</moid>

         -      </key>

         -      <startAction>powerOn</startAction>

         -      <startDelay>-1</startDelay>

         -      <startOrder>1</startOrder>

         -      <stopAction>systemDefault</stopAction>

         -      <stopDelay>-1</stopDelay>

         -      <waitForHeartbeat>systemDefault</waitForHeartbeat>

         -    </e>

            </AutoStartOrder>

            <SystemDefaults>

              <_type>vim.host.AutoStartManager.SystemDefaults</_type>

    Saving current state in /bootbank

    Clock updated.

    Time: 12:00:14   Date: 02/03/2019   UTC

Questions:

     1. What can prevent the ESXi hosts from automatically syncing it's configurations to the /bootbank/state.tgz file?

     2. If running the force backup command " /sbin/auto-backup.sh " does not resolve this, what can I try to save a "new state of the ESXi host's configurations"

     3. By chance, is there a ESXi built-in tool/command to diagnose what issues it could be having?

     4. Would Reseting the System Configuration be the next step for my issue?

          VM KB: Reset the System Configuration  ​= LINK https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.install.doc/GUID-AD450DA3-881B-4916-92...

Thank you.

Reply
0 Kudos