VMware Cloud Community
TroySchulz
Enthusiast
Enthusiast
Jump to solution

Can you recover an accidentally deleted VMDK file?

I was donated a Dell T420 server which I turned into a web server. I've installed ESXi 6 into which I created a Linux Mint LAMP server for hosting a few non profits and a few small business owners I know for free. This is a hobby level thing - not for money. I am very new to all of this.

There is a pair of 500GB SAS drives in this. One is Datastore 1 and the other Datastore 2. The web server is installed in Datastore 1.

At one point the web server crashed and it appeared I have run out of disk space so it wouldn't reboot. After reading a lot online it was suggested to see if removing one or more snapshots might not free up the space. I only had two snapshots in place. I deleted the oldest one. However, this did not appear to free up enough space. I did not want to delete my only remaining snapshot.

It was then suggested that during the process the associated VMDK files in the datastore may not have been cleaned out properly during the snapshot deletion. I browsed the datastore in which are several different files LOG files, XML files, VMSN, NVRAM, VMSD, VMX, VMXF, and a series of VMDK files, which I've been told would the be the "main" files for the virtual machine.

The files I had listed were:

Webserver0.vmdk

Webserver0-000001.vmdk

Webserver0-000002.vmdk

Webserver0-000003.vmdk

Webserver0-000004.vmdk

Webserver0-000005.vmdk

Webserver0-000006.vmdk <---This one being the newest date.

Not knowing what to do with these I right clicked on the Webserver0-000001.vmdk to see what was available and saw a dropdown menu of options. However, I may have accidentally double clicked and hit the "delete" option from the menu. Before I knew it that file was gone. I then could not boot the VM after. All I get is an error message:

Failed to power on virtual machine WebServer. File /vmfs/volumes/58866b0f-0a0319b0-38e0-e0db55027e66/WebServer/WebServer0-000001.vmdk was not found

I looked back in the datastore and now in place of that file is a Webserver0-000001-delta.vmdk file instead.

I assume all these files are required for the VM to operate? Have I just royally screwed myself and all these people I've been hosting? Is there a way to recover or rebuild that file and restore this machine? I do not have a backup since Veeam said their products won't work with free ESXi. I also have no money and cannot afford to pay a support ticket or buy recovery software that cost $699 at minimum!

I need help desperately!

29 Replies
TroySchulz
Enthusiast
Enthusiast
Jump to solution

How much free disk space do you have on the other datastore (Datastore2)? What we need is ~200GB free disk space.

I had copied a bunch of files from Datastore1 to Datastore2 previously using the Datastore Browser, but  I can delete all that to free up the space. Then I will have a full 500GB Datastore2.

remove the VM from the ESXi host's inventory (DON'T use delete from diks)

Do I just right click on the VM and choose something from the dropdown?

clone the Webserver VM's virtual disk from the source Datstore1 to the target Datsatore2 (this will consolidate all snapshots)
vmkfstools -i "/vmfs/volumes/Datastore1/Webserver/WebServer0-000006.vmdk" "/vmfs/volumes/Datastore2/Webserver/WebServer.vmdk"

Am I doing this from SSH?

copy the VM's .vmxf, and .nvram files from the old folder to the new one

Do these copy over as part of the clone process you mentioned previously?

upload the attached configuration (.vmx) file to the new folder - I've modified this file to point to the new .vmdk file name

I can just do this using the Datastore Browser?

right click the .vmx file in the new folder, and select "Add to Inventory"

Again in Datastore Browser?

PS: I assume you've already fixed the missing .vmdk descriptor file issue.

continuum said he was going to rebuild it, but I haven't seen anything back yet (unless I missed a message in the chain somewhere), so I assume he's still working on it.

Sorry for all the clarifying questions. If I sound like a complete idiot it's because I am [or I wouldn't have gotten myself into this position in the first place]. I am really new to virtualization and I'm still learning how to navigate ESXi and I certainly have no experience with SSH or any of that other stuff. So, this is a big learning experience for me. I appreciate everyone's help and patience.

0 Kudos
a_p_
Leadership
Leadership
Jump to solution

It's better to ask now than to run into issues, so don't worry about this.

I had copied a bunch of files from Datastore1 to Datastore2 previously using the Datastore Browser, but  I can delete all that to free up the space. Then I will have a full 500GB Datastore2.

Not sure what exactly you've copied, but since we are going to clone the virtual disk, the source will always be available in case the clone doesn't work as expected. So you can free up the disk space on Datstore2.

Do I just right click on the VM and choose something from the dropdown?

Yes, right click and select Remove from Inventory.

Am I doing this from SSH?

Yes, it's a command line tool. Also remember that it will run for quite some time due to the file size, so please remain patient.

Do these copy over as part of the clone process you mentioned previously?

No, that's why I have this step on the ToDo list. The vmkfstools command will only take care of the virtual disk (.vmdk) files.

I can just do this using the Datastore Browser?

Yes.

Again in Datastore Browser?

Again, Yes.

continuum said he was going to rebuild it, but I haven't seen anything back yet (unless I missed a message in the chain somewhere), so I assume he's still working on it

I doubt it, because he already replied, mentioning the rebuit file that I've already attached the rebuit file to a previous reply.

André

TroySchulz
Enthusiast
Enthusiast
Jump to solution

Yes, right click and select Remove from Inventory.

I don't see an option like that. Could it be that I should choose "Unregister"?

0 Kudos
TroySchulz
Enthusiast
Enthusiast
Jump to solution

You are absolutely awesome! It works!! I am actually in tears. You have no idea how much this means to me. What a fantastic community!!!

ErnieLouie
Contributor
Contributor
Jump to solution

Hi Andre (or should I address you as a.p.?) - I actually have done a very similar removal of a .vmdk file from the datastore like Troy and was following your thread conversation with Troy. 
I started down this path, because after I built a GNS3 environment to simulate/practice on building networks, one of the
appliances used all the disk space (or ran out of swap space), so I thought to delete some large files (older - thinking
those files are no longer used) - that was a mistake Smiley Sad
So, I saw this thread in the vmware community and wanted to pursue.  I noticed that you have helped modified the WebServer0.vmdk file (in Troy's situation) ....
this is where I got lost!  Would you be able help guide me in your thought process so I can create a new .vmdk file for this purpose, or is it
different enough that the method may not work?  I'm willing to give it a try and open for suggestions.

In Troy's VM (the Web you have made (or at least I think you have made):
1. I noticed the following changes:
CID FROM: "1065ec08" TO: "e056a0a1"
parentCID FROM "ffffffff" TO "1065ec08"
createType FROM: "vmfs" TO: "vmfsSparse"

2. You added:
parentFileNameHint "WebServer0.vmdk"

3. Under Extent description:
FROM: "RW 419430400 VMFS "WebServer0-flat.vmdk"
TO:       "419430400 VMFSSPARSE "WebServer0-000001-delta.vmdk"

4. Removed everything below the two comments:
# The Disk Data Base
#DDB

How did you determine the new values?  Or rather, how would I determine the correct modifications based on my VM files (i.e. GNS3-2.1.15.vmx)?

This is what I have:
*********************
A single DataStore [Identification = "datastore1 (2)"]  with 27GB free space.
*********************


************* Begin ***********************
output from the 'ls -elisa > filelist.txt:
*******************************************
total 125181968
264241540      8 drwxr-xr-x    1 root     root          2380 Mon Jun 10 17:44:18 2019 .
      4   1024 drwxr-xr-t    1 root     root          1960 Fri Mar 29 20:06:30 2019 ..
549454212 1213440 -rw-------    1 root     root     1241559040 Mon Jun 10 05:45:30 2019 GNS3-2.1.15-000001-delta.vmdk
553648516      0 -rw-------    1 root     root           325 Mon Jun 10 05:45:09 2019 GNS3-2.1.15-000001.vmdk
545259908   1024 -rw-------    1 root     root         27997 Wed Apr  3 01:45:37 2019 GNS3-2.1.15-Snapshot1.vmsn
322961796   1024 -rw-------    1 root     root          8684 Mon Jun 10 05:45:30 2019 GNS3-2.1.15.nvram
276824452      0 -rw-r--r--    1 root     root           615 Wed Apr  3 01:45:40 2019 GNS3-2.1.15.vmsd
268435844      8 -rwxr-xr-x    1 root     root          2577 Mon Jun 10 17:23:17 2019 GNS3-2.1.15.vmx
272630148      0 -rw-r--r--    1 root     root           266 Fri Mar 29 20:06:32 2019 GNS3-2.1.15.vmxf
557842820 21562368 -rw-------    1 root     root     22079021056 Mon Jun 10 05:45:30 2019 GNS3-2.1.15_1-000001-delta.vmdk
562037124      0 -rw-------    1 root     root           330 Mon Jun 10 05:45:20 2019 GNS3-2.1.15_1-000001.vmdk
289407364 102400000 -rw-------    1 root     root     104857600000 Wed Apr  3 01:42:10 2019 GNS3-2.1.15_1-flat.vmdk
293601668      0 -rw-------    1 root     root           508 Wed Apr  3 01:21:31 2019 GNS3-2.1.15_1.vmdk
515899780      0 -rw-r--r--    1 root     root             0 Mon Jun 10 17:44:18 2019 filelist.txt
478151044   1024 -rw-r--r--    1 root     root         42845 Mon Jun 10 05:55:16 2019 vmware-1.log
494928260   1024 -rw-r--r--    1 root     root         53217 Mon Jun 10 07:57:34 2019 vmware-2.log
511705476   1024 -rw-r--r--    1 root     root         52960 Mon Jun 10 17:23:18 2019 vmware.log
************* End **************************

************* Begin ***********************
zip file of all the "small" files from my VM (GNS3-2.1.15) including filelist.txt
*******************************************

Given this information, would you have some spare cycles in your days to help me?  If you don't think it is possible, I understand and am willing
to create this GNS3-VM environment from scratch - still a great learning experience.  Or if you beleive it is possible, but don't have the time,
that is fine as well... I'll give it a go on my own and do a little more research.  Thanks for you time.  Ernie

0 Kudos
a_p_
Leadership
Leadership
Jump to solution

I'm afraid that there's not much I can do in this case.

The first disk's base files (GNS3-2.1.15.vmdk, and GNS3-2.1.15-flat.vmdk) which are required are missing.

The VM's second virtual disk's files look ok, so it should be possible to use these files, and either extract/backup the files on it, or manually attach it to another VM (e.g. a newly created GNS3) if that helps.

André

0 Kudos
ErnieLouie
Contributor
Contributor
Jump to solution

Hi Andre - Thanks for your quick response.  I just notice, when I ssh and "ls" the GNS3-VM, I see a lot more files.  Let me see if I can download and zip for you and see if these are the missing files?

0 Kudos
ErnieLouie
Contributor
Contributor
Jump to solution

/vmfs/volumes/555a3841-37b701d0-ab79-d48564510900/GNS3-2.1.15 # ls -lag

total 125183000

drwxr-xr-x    1 root          2520 Jun 10 19:33 .

drwxr-xr-t    1 root          2100 Jun 10 20:54 ..

-rw-------    1 root     1241559040 Jun 10 05:45 GNS3-2.1.15-000001-delta.vmdk

-rw-------    1 root           325 Jun 10 05:45 GNS3-2.1.15-000001.vmdk

-rw-------    1 root         27997 Apr  3 01:45 GNS3-2.1.15-Snapshot1.vmsn

-rw-------    1 root          8684 Jun 10 05:45 GNS3-2.1.15.nvram

-rw-r--r--    1 root           615 Apr  3 01:45 GNS3-2.1.15.vmsd

-rwxr-xr-x    1 root          2577 Jun 10 19:33 GNS3-2.1.15.vmx

-rw-r--r--    1 root           266 Mar 29 20:06 GNS3-2.1.15.vmxf

-rw-------    1 root     22079021056 Jun 10 05:45 GNS3-2.1.15_1-000001-delta.vmdk

-rw-------    1 root           330 Jun 10 05:45 GNS3-2.1.15_1-000001.vmdk

-rw-------    1 root     104857600000 Apr  3 01:42 GNS3-2.1.15_1-flat.vmdk

-rw-------    1 root           508 Apr  3 01:21 GNS3-2.1.15_1.vmdk

-rw-r--r--    1 root          1793 Jun 10 17:44 filelist.txt

-rw-r--r--    1 root         42845 Jun 10 05:55 vmware-1.log

-rw-r--r--    1 root         53217 Jun 10 07:57 vmware-2.log

-rw-r--r--    1 root         52960 Jun 10 17:23 vmware-3.log

-rw-r--r--    1 root         52958 Jun 10 19:33 vmware.log

/vmfs/volumes/555a3841-37b701d0-ab79-d48564510900/GNS3-2.1.15 #

0 Kudos
ErnieLouie
Contributor
Contributor
Jump to solution

Hi Andre -  I was able to download the rest of files, please see attached.  Let me know if this is doable.  Thanks Ernie

0 Kudos
Gatete-Bruno
Contributor
Contributor
Jump to solution

Hello, 
I have been reading through this thread to try and find a solution, but I accidentally deleted my vmdk files for when the vms were initially setup, these were incremental snapshots i think, that is why i can not be able to use the currently vmdk files
I would like to get in touch with someone to assist me since am using esxi 7 with basic support but i haven't received any feedback from support

0 Kudos