5 Replies Latest reply on Feb 9, 2019 11:16 AM by fermo4learn

    [ESXi] Orphaned/Unregistered VM returns after reboot

    fermo4learn Lurker

      Could someone please assist me with this annoying issue?

      Thanks!

       

       

      System Info: 

      6.7.0 Update 1 (Build 10302608)

      Installed on a 8GB Flashdrive

      2 SSD Datastores

       

       

      Steps taken:

           1. Through the vSphere 6.7 web client, I've selected all VMs

           2. Click on "Actions", and click "Delete"

           3. VMs were removed

           4. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

           5. Selected each VM one at a time and clicked "Actions", "Unregistered" (Done this for each VM)

           6. After rebooting the ESXi Host, VM's Orphaned / Unregistered object returns with the same corresponding #

           7. Reformatted both Datastore's SSD Drives with low level format but this does not affect Orphaned / Unregistered objects

           8. Using SSH, used command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.

           9. VMs instantly get removed from the web client

           9. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

       

       

      Solution:

      Annoying but doesn't seem to take up any storage. I've reinstalled ESXi on another USB but in future cases I would like to know if there's another solution

       

       

      Questions:

           1. What causes this issue?

           2. What can I do to prevent running into this issue in the future?

           3. What can I do so that they don't reappear after rebooting the ESXi host?

       

       

      Screenshot:

        • 1. Re: [ESXi] Orphaned/Unregistered VM returns after reboot
          sjesse Master
          vExpert

          Do you also have vcenter, I see your using the host client, but if you have vcenter the vms in that inventory and it may be readding it.

          • 2. Re: [ESXi] Orphaned/Unregistered VM returns after reboot
            continuum Guru
            vExpertCommunity WarriorsUser Moderators

            I assume this is an issue with the state.tgz file.
            It would be useful to try to create a config backup manually and then inspect the actual files.

            • 3. Re: [ESXi] Orphaned/Unregistered VM returns after reboot
              fermo4learn Lurker

              sjesse I don't have vCenter setup however, I did try setting up  the vCenter Server Appliance (vCSA) as an OVF import. I don't think this is related to this issue though.

               

               

              continuum

              I've looked into the "state.tgz" file that you mentioned that I found this KB:

              Locally restoring an ESXi configuration from state.tgz backup (2043048)

               

              1. I enabled SSH on the ESXi Host
              2. WinSCP into the host
              3. At the root directory, found the soft-link that directs to  /bootbank/state.tgz -> /tmp/state.tgz
              4. I've made a backup of the "state.tgz" file and deleted it
              5. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding # and the "state.tgz" is recreated.
              6. After looking into the "state.tgz" compressed file I found a file that contained a list of my previous VMs - state.tgz\state\state\local\local\etc\vmware\hostd\vmInventory.xml
              7. I found the same file in my ESXi host under - ./etc/vmware/hostd/
              8. I've removed the entries but upon restarting the ESXi host the VM's return

               

               

              Question:

              1.  Is the "state.tgz" file basically a way "to restore the locally saved ESXi configuration" if I were to reinstall the ESXi host onto another flash drive?

              • 4. Re: [ESXi] Orphaned/Unregistered VM returns after reboot
                wila Guru
                vExpertCommunity WarriorsUser Moderators

                Hi,

                 

                How long do you wait between deleting/unregistering those VMs and rebooting the host?

                 

                I'm asking this as with vSphere/ESXi you are basically running from a RAM disk. Something that most of the times is a very good thing, other times however it can bite you, especially if you do not realize that this is the case.

                Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.

                 

                You can force this by running the following command manually.

                /sbin/auto-backup.sh

                 

                Hope this helps,

                --

                Wil

                | Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
                | Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
                | More info at https://www.vimalin.com
                | Twitter @wilva
                | VMware Wiki at http://www.vi-toolkit.com
                • 5. Re: [ESXi] Orphaned/Unregistered VM returns after reboot
                  fermo4learn Lurker

                  Hi wila

                   

                  I've attempted to troubleshooting with taking information you provided in consideration as well as allowing enough time for the ESXi host to self update if it's already set to be "auto backup" by itself. These steps have been repeated over a week now.

                   

                   

                  Update:

                       1. Using SSH, used the command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.

                       2. Soft rebooted the ESXi host through the web client

                       3. After rebooting the ESXi host, all VM's Orphaned / Unregistered object return with a corresponding #

                       4. Also, a previously restored VM was lost and was not listed as a new detected Orphaned / Unregistered VM

                       5. Luckily I was able to restored the VM by registering and finding the location path of it's .VMDK files

                       6. Using SSH, used the command   /sbin/auto-backup.sh and allowed enough time for the ESXi to

                  Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.

                       7. After using SSH again to unregister VM's and soft rebooting the ESXi host, it reverts back to the corrupted state ( 3. Orphaned / Unregistered )

                   

                  CMD log: /sbin/auto-backup.sh

                  [root@bvzesxi:~] /sbin/auto-backup.sh

                           --- /etc/vmware/hostd/vmAutoStart.xml

                           +++ /tmp/auto-backup.2100205//etc/vmware/hostd/vmAutoStart.xml

                           @@ -1,20 +1,7 @@

                            <ConfigRoot>

                              <AutoStartOrder>

                           -    <_length>1</_length>

                           +    <_length>0</_length>

                                <_type>vim.host.AutoStartManager.AutoPowerInfo[]</_type>

                           -    <e id="0">

                           -      <_type>vim.host.AutoStartManager.AutoPowerInfo</_type>

                           -      <key>

                           -        <_type>vim.VirtualMachine</_type>

                           -        <moid>22</moid>

                           -      </key>

                           -      <startAction>powerOn</startAction>

                           -      <startDelay>-1</startDelay>

                           -      <startOrder>1</startOrder>

                           -      <stopAction>systemDefault</stopAction>

                           -      <stopDelay>-1</stopDelay>

                           -      <waitForHeartbeat>systemDefault</waitForHeartbeat>

                           -    </e>

                              </AutoStartOrder>

                              <SystemDefaults>

                                <_type>vim.host.AutoStartManager.SystemDefaults</_type>

                      Saving current state in /bootbank

                      Clock updated.

                      Time: 12:00:14   Date: 02/03/2019   UTC

                   

                   

                   

                  Questions:

                       1. What can prevent the ESXi hosts from automatically syncing it's configurations to the /bootbank/state.tgz file?

                       2. If running the force backup command " /sbin/auto-backup.sh " does not resolve this, what can I try to save a "new state of the ESXi host's configurations"

                       3. By chance, is there a ESXi built-in tool/command to diagnose what issues it could be having?

                       4. Would Reseting the System Configuration be the next step for my issue?

                            VM KB: Reset the System Configuration  = LINK https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.install.doc/GUID-AD450DA3-881B-4916-9231-9B6998AAC3AA.html

                   

                   

                  Thank you.