10 Replies Latest reply on Feb 12, 2020 4:21 PM by continuum

    [HELP] Missing VMs folders from datastore

    hede0n Lurker

      Hi

       

      I think I had power down when I was at work, and after rebooting ESXI 5.5 all my VMs were UNKNOWN (INACCESSIBLE). Tried to restore them to find out that all folders with VMs are gone from Datastore, with all virtual disks, config files, etc.

      There are two folders instead:

       

      .sdd.sf

       

      and

       

      vsantraces (which I think was created on the day of power loss)

       

      Could somebody advise me how to get them back?

        • 1. Re: [HELP] Missing VMs folders from datastore
          hede0n Lurker

          Little update:

           

          Accessed datastore through SSH and while issuing ls -l there is a reference to missing folders in form on listing them like this:

           

          [QUOTE]

          ls: ./SAS-University-Edition: No such file or directory

          ls: ./Mint: No such file or directory

          ls: ./alwaysVPN: No such file or directory

          ls: ./ZoneMinderServer: No such file or directory

          ls: ./ArchVPN: No such file or directory

          ls: ./ArchSamba: No such file or directory

          ls: ./ArchPlexServer: No such file or directory

          ls: ./HomeAutomotion: No such file or directory

          ls: ./NODE-RED: No such file or directory

          total 8

          drwxr-xr-x    1 root     root          1680 Apr  9 17:11 vsantraces

           

          [/QUOTE]

           

          I am not linux guru, but I think linux does not list deleted folders like that, so there is a chance something is still there.

           

          Any ideas how to recover it?

          • 2. Re: [HELP] Missing VMs folders from datastore
            weinstein5 Guru

            Welcome to the Community - I have moved your question to a more appropriate forum

            • 3. Re: [HELP] Missing VMs folders from datastore
              vfk Expert

              Hi, do you have access to the datastore where the VM are hosted?  Are all the datastore mounted?  Can you provide the output for this command?

               

              esxcli storage filesystem list

              • 4. Re: [HELP] Missing VMs folders from datastore
                hede0n Lurker

                I believe so. Missing files and folders were on small_datastore. It is like all of them got corrupted. I can access small_datastore and create new folders, upload files etc...

                 

                Output of said command:

                 

                ~ # esxcli storage filesystem list

                Mount Point                                        Volume Name      UUID                                 Mounted  Type             Size          Free

                -------------------------------------------------  ---------------  -----------------------------------  -------  ------  -------------  ------------

                /vmfs/volumes/545f6d74-0dff7468-c23e-0090f5dfcce7  small_datastore  545f6d74-0dff7468-c23e-0090f5dfcce7     true  VMFS-5   999922073600  176048570368

                /vmfs/volumes/545f6d8a-1cfca064-ee41-0090f5dfcce7  large_datastore  545f6d8a-1cfca064-ee41-0090f5dfcce7     true  VMFS-5  2000112582656    3584032768

                /vmfs/volumes/9ac672ef-36d6e423-684d-31a645113840                   9ac672ef-36d6e423-684d-31a645113840     true  vfat        261853184      96837632

                /vmfs/volumes/66ccf2ee-b1ba4ac5-3cd2-e8ddb6c2b71c                   66ccf2ee-b1ba4ac5-3cd2-e8ddb6c2b71c     true  vfat        261853184     261844992

                /vmfs/volumes/545f6543-c587f874-e80a-0090f5dfcce7                   545f6543-c587f874-e80a-0090f5dfcce7     true  vfat        299712512      96919552

                ~ #

                • 5. Re: [HELP] Missing VMs folders from datastore
                  vfk Expert

                  So the datastore in mounted, you can write stuff into it, but you cannot see the old stuff, can you see at least the folder structure of the VM?

                  • 6. Re: [HELP] Missing VMs folders from datastore
                    hede0n Lurker

                    Thanks for reply. I am not entirely sure if I know what you are asking about. Before incident I have never ssh'd to the host, so not sure what folder structure was there but the one I am seeing now seams to aright... Could you please specify?

                    • 7. Re: [HELP] Missing VMs folders from datastore
                      eptc Novice

                      Hi,

                      Did you get this issue resolved?

                      I am having the same problem. My DataStore looks empty but almost 1TB of capacity is used.

                       

                      Any advice on this would be most appreciated.

                       

                      D

                      • 8. Re: [HELP] Missing VMs folders from datastore
                        continuum Guru
                        Community WarriorsvExpert

                        Hi
                        stop using that datastore immediatly - anything you may do on that datastore may overwrite existing VMs.
                        Next create a vmfs-header-dump - see Collecting and applying raw metadata dumps on VMFS volumes using DD (Data Description) (1020645) | VMware KB

                        If you gve me a download for the header-dump I can probably help you.
                        Ulli

                        • 9. Re: [HELP] Missing VMs folders from datastore
                          pierree Lurker

                          also have this prob too, running esxi6

                          Can't figure out….

                          • 10. Re: [HELP] Missing VMs folders from datastore
                            continuum Guru
                            vExpertCommunity Warriors

                            Hi

                            I may be able to help you but with VMFS 6 results are not predictable before I have seen a VMFS header dump - see
                            https://vm-sickbay.com/create-a-vmfs-header-dump-using-an-esxi-host-in-production

                            Keep in mind that keeping the affected datastore in production (in case that is still possible( will soon erase all traces of the lost data.

                            So either act quick or stop all activity asap.

                            I need to know what to look for - this can be a list of VMs, a list of flat,vmdks - the better you describe what disappeared the better the chances.

                            Contact me via skype to discuss the details.

                            Ulli