5 Replies Latest reply on Feb 9, 2019 11:16 AM by fermo4learn

[ESXi] Orphaned/Unregistered VM returns after reboot

fermo4learn Jan 29, 2019 2:46 PM

Could someone please assist me with this annoying issue?

Thanks!

System Info:

6.7.0 Update 1 (Build 10302608)

Installed on a 8GB Flashdrive

2 SSD Datastores

Steps taken:

1. Through the vSphere 6.7 web client, I've selected all VMs

2. Click on "Actions", and click "Delete"

3. VMs were removed

4. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

5. Selected each VM one at a time and clicked "Actions", "Unregistered" (Done this for each VM)

6. After rebooting the ESXi Host, VM's Orphaned / Unregistered object returns with the same corresponding #

7. Reformatted both Datastore's SSD Drives with low level format but this does not affect Orphaned / Unregistered objects

8. Using SSH, used command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.

9. VMs instantly get removed from the web client

9. After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding #

Solution:

Annoying but doesn't seem to take up any storage. I've reinstalled ESXi on another USB but in future cases I would like to know if there's another solution

Questions:

1. What causes this issue?

2. What can I do to prevent running into this issue in the future?

3. What can I do so that they don't reappear after rebooting the ESXi host?

Screenshot:

1. Re: [ESXi] Orphaned/Unregistered VM returns after reboot

sjesse Jan 29, 2019 3:48 PM (in response to fermo4learn)

Do you also have vcenter, I see your using the host client, but if you have vcenter the vms in that inventory and it may be readding it.
Like Show 0 Likes(0)

Actions
2. Re: [ESXi] Orphaned/Unregistered VM returns after reboot

continuum Jan 29, 2019 5:52 PM (in response to fermo4learn)

I assume this is an issue with the state.tgz file.
It would be useful to try to create a config backup manually and then inspect the actual files.
Like Show 0 Likes(0)

Actions
3. Re: [ESXi] Orphaned/Unregistered VM returns after reboot

fermo4learn Jan 31, 2019 12:32 PM (in response to fermo4learn)
sjesse I don't have vCenter setup however, I did try setting up the vCenter Server Appliance (vCSA) as an OVF import. I don't think this is related to this issue though.

continuum
I've looked into the "state.tgz" file that you mentioned that I found this KB:
Locally restoring an ESXi configuration from state.tgz backup (2043048)

I enabled SSH on the ESXi Host
WinSCP into the host
At the root directory, found the soft-link that directs to /bootbank/state.tgz -> /tmp/state.tgz
I've made a backup of the "state.tgz" file and deleted it
After rebooting the ESXi Host, VM's Orphaned / Unregistered object return with a corresponding # and the "state.tgz" is recreated.
After looking into the "state.tgz" compressed file I found a file that contained a list of my previous VMs - state.tgz\state\state\local\local\etc\vmware\hostd\vmInventory.xml
I found the same file in my ESXi host under - ./etc/vmware/hostd/
I've removed the entries but upon restarting the ESXi host the VM's return

Question:
1. Is the "state.tgz" file basically a way "to restore the locally saved ESXi configuration" if I were to reinstall the ESXi host onto another flash drive?
Like Show 0 Likes(0)

Actions
4. Re: [ESXi] Orphaned/Unregistered VM returns after reboot

wila Feb 1, 2019 5:11 AM (in response to fermo4learn)

Hi,

How long do you wait between deleting/unregistering those VMs and rebooting the host?

I'm asking this as with vSphere/ESXi you are basically running from a RAM disk. Something that most of the times is a very good thing, other times however it can bite you, especially if you do not realize that this is the case.
Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.

You can force this by running the following command manually.
/sbin/auto-backup.sh

Hope this helps,
--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Desktop Products
| Vimalin : Automated backups for VMware Fusion and VMware Workstation Professional
| More info at https://www.vimalin.com
| Twitter @wilva
| VMware Wiki at http://www.vi-toolkit.com
Like Show 0 Likes(0)

Actions
5. Re: [ESXi] Orphaned/Unregistered VM returns after reboot

fermo4learn Feb 9, 2019 11:16 AM (in response to wila)

Hi wila

I've attempted to troubleshooting with taking information you provided in consideration as well as allowing enough time for the ESXi host to self update if it's already set to be "auto backup" by itself. These steps have been repeated over a week now.

Update:
     1. Using SSH, used the command vim-cmd /vmsvc/unregister <VMID> = vim-cmd /vmsvc/unregister 1, 2, 3, etc.
     2. Soft rebooted the ESXi host through the web client
     3. After rebooting the ESXi host, all VM's Orphaned / Unregistered object return with a corresponding #
     4. Also, a previously restored VM was lost and was not listed as a new detected Orphaned / Unregistered VM
     5. Luckily I was able to restored the VM by registering and finding the location path of it's .VMDK files
     6. Using SSH, used the command   /sbin/auto-backup.sh and allowed enough time for the ESXi to
Changes made to the configuration are automatically synced to things like the state file, but only 1 time per hour.
     7. After using SSH again to unregister VM's and soft rebooting the ESXi host, it reverts back to the corrupted state ( 3. Orphaned / Unregistered )

CMD log: /sbin/auto-backup.sh
[root@bvzesxi:~] /sbin/auto-backup.sh
         --- /etc/vmware/hostd/vmAutoStart.xml
         +++ /tmp/auto-backup.2100205//etc/vmware/hostd/vmAutoStart.xml
         @@ -1,20 +1,7 @@
          <ConfigRoot>
            <AutoStartOrder>
         -    <_length>1</_length>
         +    <_length>0</_length>
              <_type>vim.host.AutoStartManager.AutoPowerInfo[]</_type>
         -    <e id="0">
         -      <_type>vim.host.AutoStartManager.AutoPowerInfo</_type>
         -      <key>
         -        <_type>vim.VirtualMachine</_type>
         -        <moid>22</moid>
         -      </key>
         -      <startAction>powerOn</startAction>
         -      <startDelay>-1</startDelay>
         -      <startOrder>1</startOrder>
         -      <stopAction>systemDefault</stopAction>
         -      <stopDelay>-1</stopDelay>
         -      <waitForHeartbeat>systemDefault</waitForHeartbeat>
         -    </e>
            </AutoStartOrder>
            <SystemDefaults>
              <_type>vim.host.AutoStartManager.SystemDefaults</_type>
    Saving current state in /bootbank
    Clock updated.
    Time: 12:00:14   Date: 02/03/2019   UTC

Questions:
     1. What can prevent the ESXi hosts from automatically syncing it's configurations to the /bootbank/state.tgz file?
     2. If running the force backup command " /sbin/auto-backup.sh " does not resolve this, what can I try to save a "new state of the ESXi host's configurations"
     3. By chance, is there a ESXi built-in tool/command to diagnose what issues it could be having?
     4. Would Reseting the System Configuration be the next step for my issue?
          VM KB: Reset the System Configuration = LINK https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.esxi.install.doc/GUID-AD450DA3-881B-4916-9231-9B6998AAC3AA.html

Thank you.
Like Show 0 Likes(0)

Actions