Contributor
Contributor

Validating Snapshot Removal

Jump to solution

OK, I've been an idiot. In my ignorance, I have been happily creating snapshots against a Windows 2008 Server running as a virtual machine for the past 12 months or so and had ended up in a chain/hierarchy of about 8 snapshots.

This morning, I decided (after reading some articles on the nature of snapshots) I could not go on like this so decided to delete all the snapshots via the Snapshot Manager. I powered down the virtual server and clicked the 'Delete All Snapshots' button. After a couple of hours the whole box seemed to lock - I couldn't get access via the Infrastructure Client, or even from the console on the physical server. I waited another hour and then performed a hard reboot (in hindsight, I suspect I was too impatient).

Expecting the worst, I rebooted the ESXi 3.5.0 server and, to my surprise it booted happily and also, to my even greater surprise, the Windows2008 virtual machine bootedwithout any error too. I checked the Snapshot Manager and sure enough it isnot showing any snapshots. So, on the face of it, everything looks OK, but when I look in the datastore for the virtual machine, I still see numerous .vmdk files (see attached screenshot).

Can anyone tell me if these files are to be expected and, perhaps more importantly if there is any way to check/validate that the snapshot removal has been successful and, if not, is there a way to correct the problem I may have created for myself?

Thanks.

Tags (2)
0 Kudos
1 Solution

Accepted Solutions
Virtuoso
Virtuoso

You can also access the console via the method detailed in the below link

http://vmprofessional.com/2008/06/esxi-unsupported-console-access.html

View solution in original post

0 Kudos
19 Replies
Virtuoso
Virtuoso

Your snapshot files have not been deleted. One easy way to check if the snapshot files are being referenced is to edit the settings of the VM and select the hard disk - have a look at the file name(s) the hard disk(s) are referencing.

How much free disk space is there on this particular datastore?

0 Kudos
Contributor
Contributor

Hi, I did as you suggested and it looks like the VM is still referencing the snapshot files (e.g. DataStore1 is referencing Zeus-000006.vmdk.) I'm assuming that this means that Zeus-000005.vmdk, Zeus-000004.vmdk are all needed as well?).

I am currently moving other files (not associated with this VM) of the physical DataStorewhich is currently saying that only 6MB is free (even tough I have already moved 32GB of the drive, so I'm not sure what is going on there?)

I read elswhere that, assuming the original operation failed due to lack of disk space, that if I create a new snapshot and then perform a 'delete all' against that it will clean up all of the 'orphaned' snapshot files in the process - do you think this is likley?

0 Kudos
Champion
Champion

Seems you are in a bad situation... follow the KB http://kb.vmware.com/kb/1007849 (in your case you will have to use the RCLI tool, look for it at the vmware webpage and install it).

Any further help you need on this please let us know.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Linux Server Senior Administrator

Marcelo Soares - VCP 3/4/5, VCI 5, VCAP5-DCA, vExpert 2012/2013 - Twitter: @mtsoares42
Virtuoso
Virtuoso

Do you have any other datastores? Can you create a new datastore? Do you have a server imaging tool? vCenter?

Creating a new snapshot and performing a delete all will not help you in your case as you don't have enough free disk space.

0 Kudos
Contributor
Contributor

I do have another (empty) datastore that has a capacity of 232GB. I'm afarid I don't have a server imaging tool/vCenter.

The ESXi server has three data stores each of 232GB. The faulty VM machine is stored on the first datastore.

0 Kudos
Virtuoso
Virtuoso

Perform the steps indicated in the kb article the other poster provided a link to - clone the disk to the other datastore when performing the steps.

Contributor
Contributor

OK, I now have the two virtual machines back up and running and I have managed to consolidate the snapshots from the lighter of the two VMs.

I still have seven snapshots of the primary server,details as follow:

Total DataStore size: 228GB

Free space: 29.95GB

Approximate SnapShot file sizes: 95GB (see attached)

I also have a free/empty data store available on the ESXi host of 228GB.

Having been burnt, I would like to consolidate these snapshots. Is there a technique/tool that will enable me to do this with the resources I have available?

Many thanks for all your help by the way - much appreciated.

0 Kudos
Virtuoso
Virtuoso

Normally you would be able to delete one snapshot at a time in your current situation, however as you don't see the snapshots within snapshot manager and due to having quite a number of them and not much free disk space....... and no imaging tool or vCenter....... you'll have to follow the steps as per the kb article cloning the disk to another datastore with more free space.

I'm off to bed now but am happy to respond to further questions in the morning if no-one else has stepped in for the mean time.

Good luck!

0 Kudos
Contributor
Contributor

Thank you for all your help today - very much appreciated. I've reviewed the KB article and will attempt the clone once I have created a double backup of the VM. I do have one very basic question though... How to you get console access to the host ESXi server (as in the video). I've read about the CLI VMware RCLI but, reading the notes, I'm not sure this is the same thing. Perhaps you could point me in the right direction?

Enjoy your sleep!

0 Kudos
Contributor
Contributor

I think I've sussed it - I need to run the Remote CLI Appliance as a VM on the server

0 Kudos
Virtuoso
Virtuoso

You can also access the console via the method detailed in the below link

http://vmprofessional.com/2008/06/esxi-unsupported-console-access.html

View solution in original post

0 Kudos
Contributor
Contributor

I think my Linux understanding, or more precisely lack of understanding, is becoming a major hurdle!

Following the video/instructions...

I tried logging on to the host console (via the 'unsupported' route, but once I had logged in I couldn't run the tools I needed (like 'vmware-cmd' to list the directories containing the virtual machines). I then panicked as Ididn't know how to correctly logout and get back to the usual host screen, so I typed in 'exit' and hoped for the best!)

I then reverted to plan 'A' and fired up the Remote CLI Appliance and logged in. This time when I typed in 'vmware-cmd -l' I got the expected result (as in the video) and it listed the three VMs running on the ESXi host. I then tried to change directory to the VM of interest but got the message "No such file or directory".

Now, I'm guessing that this is because I'm logged into the Remote CLI Appliance and not the host ESXi server, but now I'm stuck:

How can I actually get to the directory of my target VM machine (the one I want to try and fix), and if I do, will I have the relevant support commands available to me or will I just have the same problem as when I logged onto the host console directly?

Even assuming someone can help me over the above problems, I still have an issue as in the video, the snapshots files have a descriptor file and an delta file, my snapshots only appear to consist of one file - have these been combined in ESXi and can I treat them as one consolidated file or am I missing something (else)?

Sorry for some many questions, but the more questions I answer, the more questions seem to crop up.

Can anyone put me back on track?

Thanks.

0 Kudos
Champion
Champion

Via the unsupported method you will not be able to execute vmkfstools, I believe... so... goingo to the CLI:

You don't have to navigate to the directory, as you are not on shell. Just tell the vmkfstools full paths, like:

vmkfstools -i /vmfs/volumes/[datastore_name]/[vm_folder]/xxxxxxxxx-000001.vmdk /vmfs/volumes/[datastore_name]/[vm_folder]/newdisk.vmdk

Don't worry also about what the datastore browser shows you. The snapshots are really compound of 2 files (as virtual disks) but it only shows you the name you "need" to know. Use the one that appears.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Linux Server Senior Administrator

Marcelo Soares - VCP 3/4/5, VCI 5, VCAP5-DCA, vExpert 2012/2013 - Twitter: @mtsoares42
0 Kudos
Contributor
Contributor

Thanks Marcelo,

I can see how this would work for the ToolKit commands like 'vmkfstools' and 'vmware-cmd' etc, but how about the generic Linux commands against the files? For example, the KB tells you to check the snapshot chain CIDs/Parent CIDs by running the following command against the snapshot descriptor file:

less myVM- 000003.vmdk | egrep -i "cid|parentcid|vmdk" --color

But even if I specifiy the full path of the myVM-000003.vmdk file (/vmfs/volumes/.../myVM-000003.vmdk) I still get a "File not found" message (as I'm assuming the shell doesn't know anything about the ESXi host?).

Thanks again

Neil.

0 Kudos
Champion
Champion

Hmmmmm... installing here the CLI, not sure about the sintax. I think you need to spacify the file like:

VMdir/disk-000001.vmdk

Try something like that. If you go to the Edit Settings of any VM you will see when clicking a disk the full path above. This one I think is needed.

The CID check you can do simply by opening the vmdk header file (not the delta) with vi using the unsupported mode. But I would not worry about it because it does not matter too much, only if you messed up with your files.

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Linux Server Senior Administrator

Marcelo Soares - VCP 3/4/5, VCI 5, VCAP5-DCA, vExpert 2012/2013 - Twitter: @mtsoares42
0 Kudos
Champion
Champion

The following worked for me:

C:\Program Files\VMware\VMware vSphere CLI\bin>vmkfstools.pl --server 10.10.10.10 -h host1.local -i win2008/win2008.vmdk win2008/win2008_2.vmdk

Marcelo Soares

VMWare Certified Professional 310/410

Technical Support Engineer

Linux Server Senior Administrator

Marcelo Soares - VCP 3/4/5, VCI 5, VCAP5-DCA, vExpert 2012/2013 - Twitter: @mtsoares42
0 Kudos

Hi,

If cloning poses a problem, then just forget about the cloning altogether and simply use VMware converter. If you start converter, and choose to convert a physical machine, you can point to the running VM and just P2V it (actually V2V it) a datastore on the ESXi host. No console fiddling required Smiley Happy

Wrote about that once in "ye olde snapshot" : http://www.vmdamentals.com/?p=76



Visit my blog at http://www.vmdamentals.com

Visit my blog at http://www.vmdamentals.com
0 Kudos
Contributor
Contributor

Thank you Erik,

Just to show/prove my ignorance of all this I had not even heard of the VMWare Converter. It does sound like a 'safe' option as it means I don't have to 'fiddle' with the ESXi at the file level. I'll read the Converter documentation and give it a go.

Just one question, is the end result of this process the same as if I had built the VM from scratch? I guess what I'm asking is is there a cost/downside to the convenience?

Thanks again.

Neil.

0 Kudos
Contributor
Contributor

Ah yes, this works for me too - thanks very much. I'm probably going to leave this now until the weekend as this is a production server and I want to make sure it is well backed up before I actually 'do anything'. I'll post next week with an update.

The support I've received from this has been great - as you've undoubtedly gathered I am new to VMWare and I would have been completely lost without this facility!

Neil.

0 Kudos