VM Snapshots are not deleted properly

[tipsy01]

We are running a VI3 infrastructure and having issues with virtual machine snapshots.

We are able to create snapshots, however on one particular we are unable to delete them through the VI Client. We keep getting "Operation timed out" from the task. Sometimes we are able to delete the snapshot successfully but although the snapshot disappears from the snapshot manager the actual disk files remain on the datastore.

We are now several (6+) snapshots down the line, the VI client knows nothing about some of the snapshots and still with the VMDK files inhabiting one of our datastores. However we are running low on disk space.

My question is what files make up a snapshot? Can I just delete these files from the datastore if I leave the most recenet snapshot and original disks / configuration files in place?

Any help would be appreciated.

[LarsLiljeroth]

Try this:

First make a backup then

vmware-cmd removesnapshots

/ Lars

[tipsy01]

Hi Lars,

I tried this but doesn't seem to make much difference.

[LarsLiljeroth]

What is the output from you command ... How long time does it take ? Ussually it takes some time. ( minutes )

[tipsy01]

It ran for about 20 mins with no output so we assumed it had stopped. Should we run this overnight to give sufficient time to finish?

[LarsLiljeroth]

A successful remove has the output = removesnapshots() = 1

So if you recieved no output i don't know.

How large is the snapshot ? ( servername-delta.vmdk )

[tipsy01]

We have 9 delta.vmdk files ranging from 16MB to 10GB in size (half of these are 3GB or over)

[LarsLiljeroth]

So any changes here ?

I think the process is going on in the background.

What files do you see as the last accessed ? the delta or the "flat"

[tipsy01]

When I thought it had stopped I close the putty session I had open.

I have now restarted the process and I'll leave the putty session open until I get some output hopefully later tonight.

[LarsLiljeroth]

For snapshot files of that size yes i would say hours... maybe 4-5

But give it a try.

[tipsy01]

That command does not seem to have worked. It is still running with no output. The trouble is we are rapidly running out of space on that particular datastore. Is there another way we can remove the snapshots?

[LarsLiljeroth]

Not to my knowledge I would create a SR asap..

[kastro · Accepted solution]

If you dont have enough free space on datastore then the only way to get rid of snapshot is to power off VM and make clone of this VM on another datastore (assuming that you have enough free space to do so). Cloned VM will be exactly the same as original, only snapshot files will be consolidated in VMDK. After that you can delete original VM with snapshots.

[tipsy01]

Thanks for the input so far guys I'm trying to clone the vm now. I'll let you know how it goes.

[ezed]

we have had similar deals, time is the key to allowing for snapshots to commit, could be several hours on large snaps. Cloning or backup and restore are often better options with large snapshots (snaps over a gig are a bit dangerous). there is a good way to track whether a delta is in use

grep the flat files for CID: the following is sample output. CID is the disk ID, parentCID is the disk that the delta is based on, all f's indicate the root disk, anything else will refer to another delta, you can follow the chain.

CID=04ee60c7

parentCID=ffffffff

[depping]

This will take a long time because the snapshots are nested and will be merged to the earlier snapshot. this is also why you are losing more diskspace. Check my blog: http://www.yellow-bricks.com/2008/01/07/delete-all-snapshots/

Duncan

My virtualisation blog: