VMware Cloud Community
hede0n
Contributor
Contributor

[HELP] Missing VMs folders from datastore

Hi

I think I had power down when I was at work, and after rebooting ESXI 5.5 all my VMs were UNKNOWN (INACCESSIBLE). Tried to restore them to find out that all folders with VMs are gone from Datastore, with all virtual disks, config files, etc.

There are two folders instead:

.sdd.sf

and

vsantraces (which I think was created on the day of power loss)

Could somebody advise me how to get them back?

12 Replies
hede0n
Contributor
Contributor

Little update:

Accessed datastore through SSH and while issuing ls -l there is a reference to missing folders in form on listing them like this:

[QUOTE]

ls: ./SAS-University-Edition: No such file or directory

ls: ./Mint: No such file or directory

ls: ./alwaysVPN: No such file or directory

ls: ./ZoneMinderServer: No such file or directory

ls: ./ArchVPN: No such file or directory

ls: ./ArchSamba: No such file or directory

ls: ./ArchPlexServer: No such file or directory

ls: ./HomeAutomotion: No such file or directory

ls: ./NODE-RED: No such file or directory

total 8

drwxr-xr-x    1 root     root          1680 Apr  9 17:11 vsantraces

[/QUOTE]

I am not linux guru, but I think linux does not list deleted folders like that, so there is a chance something is still there.

Any ideas how to recover it?

Reply
0 Kudos
weinstein5
Immortal
Immortal

Welcome to the Community - I have moved your question to a more appropriate forum

If you find this or any other answer useful please consider awarding points by marking the answer correct or helpful
vfk
Expert
Expert

Hi, do you have access to the datastore where the VM are hosted?  Are all the datastore mounted?  Can you provide the output for this command?

esxcli storage filesystem list

--- If you found this or any other answer helpful, please consider the use of the Helpful or Correct buttons to award points. vfk Systems Manager / Technical Architect VCP5-DCV, VCAP5-DCA, vExpert, ITILv3, CCNA, MCP
Reply
0 Kudos
hede0n
Contributor
Contributor

I believe so. Missing files and folders were on small_datastore. It is like all of them got corrupted. I can access small_datastore and create new folders, upload files etc...

Output of said command:

~ # esxcli storage filesystem list

Mount Point                                        Volume Name      UUID                                 Mounted  Type             Size          Free

-------------------------------------------------  ---------------  -----------------------------------  -------  ------  -------------  ------------

/vmfs/volumes/545f6d74-0dff7468-c23e-0090f5dfcce7  small_datastore  545f6d74-0dff7468-c23e-0090f5dfcce7     true  VMFS-5   999922073600  176048570368

/vmfs/volumes/545f6d8a-1cfca064-ee41-0090f5dfcce7  large_datastore  545f6d8a-1cfca064-ee41-0090f5dfcce7     true  VMFS-5  2000112582656    3584032768

/vmfs/volumes/9ac672ef-36d6e423-684d-31a645113840                   9ac672ef-36d6e423-684d-31a645113840     true  vfat        261853184      96837632

/vmfs/volumes/66ccf2ee-b1ba4ac5-3cd2-e8ddb6c2b71c                   66ccf2ee-b1ba4ac5-3cd2-e8ddb6c2b71c     true  vfat        261853184     261844992

/vmfs/volumes/545f6543-c587f874-e80a-0090f5dfcce7                   545f6543-c587f874-e80a-0090f5dfcce7     true  vfat        299712512      96919552

~ #

Reply
0 Kudos
vfk
Expert
Expert

So the datastore in mounted, you can write stuff into it, but you cannot see the old stuff, can you see at least the folder structure of the VM?

--- If you found this or any other answer helpful, please consider the use of the Helpful or Correct buttons to award points. vfk Systems Manager / Technical Architect VCP5-DCV, VCAP5-DCA, vExpert, ITILv3, CCNA, MCP
Reply
0 Kudos
hede0n
Contributor
Contributor

Thanks for reply. I am not entirely sure if I know what you are asking about. Before incident I have never ssh'd to the host, so not sure what folder structure was there but the one I am seeing now seams to aright... Could you please specify?

Reply
0 Kudos
eptc
Contributor
Contributor

Hi,

Did you get this issue resolved?

I am having the same problem. My DataStore looks empty but almost 1TB of capacity is used.

Any advice on this would be most appreciated.

D

Reply
0 Kudos
continuum
Immortal
Immortal

Hi
stop using that datastore immediatly - anything you may do on that datastore may overwrite existing VMs.
Next create a vmfs-header-dump - see Collecting and applying raw metadata dumps on VMFS volumes using DD (Data Description) (1020645) | V...

If you gve me a download for the header-dump I can probably help you.
Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
pierree
Contributor
Contributor

also have this prob too, running esxi6

Can't figure out….

Reply
0 Kudos
continuum
Immortal
Immortal

Hi

I may be able to help you but with VMFS 6 results are not predictable before I have seen a VMFS header dump - see
https://vm-sickbay.com/create-a-vmfs-header-dump-using-an-esxi-host-in-production

Keep in mind that keeping the affected datastore in production (in case that is still possible( will soon erase all traces of the lost data.

So either act quick or stop all activity asap.

I need to know what to look for - this can be a list of VMs, a list of flat,vmdks - the better you describe what disappeared the better the chances.

Contact me via skype to discuss the details.

Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos
JerrP
Contributor
Contributor

Hello Continuum, I am having a similar issue to this thread.  I was wondering if you would be willing to assist me with recovering the vm data.

ESXi local VMFS5 volume.  Two active VM's went offline and all vm's became listed as unknown.  I can browse all folders except the for the two vm's that went offline. All activity on the drive was stopped so nothing has been written over.

New NAS has been installed and is ready to copy data over.

Let me know if you can help.

Thanks.

J.

Reply
0 Kudos
continuum
Immortal
Immortal

I need a VMFS header dump - see Create a VMFS-Header-dump using an ESXi-Host in production | VM-Sickbay

Ulli


________________________________________________
Do you need support with a VMFS recovery problem ? - send a message via skype "sanbarrow"
I do not support Workstation 16 at this time ...

Reply
0 Kudos