VMware Cloud Community
Kosch
Contributor
Contributor
Jump to solution

Snap Shot Oddities

Hi Guys

We have ESX 3.02 (52542)

Got a odd situation here.

After running a cmd line I was provided I was able to identify two guests that had some strange snapshots.

Server A (Live) & Server B (Dev) have snapshot delta files present but VC does not show them.

On Server A it looks like the snapshots where taken and then recommitted to the original disk but the delta files werent deleted. Although from past experiance I thought the delta files shrunk as the data was consolidated back into the orignal vmdk?

From watching the delta files via putty for Server A I can see that they do not grow when I copy 700mb of logs to the server. Also inspecting the .vmx file for Server A it does not reference any of the delta files as you would expect it to. The delta files last mod date shows 29.05.08.

When I watch the delta files for Server B I can see that they do indeed grow when I copy new files to the disks and the vmx file does reference the delta files

As Server B is a development guest of mine and the data being test I switched the VM off, deleted one delta file via VC datastore browser for the 3rd hdd, removed the old hdd via vc and then re-added it & the machine booted fine. Not the best way to fix its delta files as I lost some rubbish data but wanted to see if it could be done that way on a dev box

Is there a command line I can use to try and reconsolidate the snapshots on Server B?

I'm not quite sure what to do with Server A as its a production VM but I suppose rather than delete the delta outright it might be best to move it to another LUN and see what happens as at least that way we can always move it back if there are problems.

Anyone come across this before or offer any advice?

Thanks

Doug

Reply
0 Kudos
1 Solution

Accepted Solutions
Pizzaman
Contributor
Contributor
Jump to solution

We have had similar problems here in the past, below is a document I created for internal use if it happened while I am not here.

Forced snapshoot removal.

Step 1 (backup, or can use anther method you are confident of restoring from)

Turn off machine

Make a backup of machine, this can be done with following command from a putty session (alter path)

cp -r /vmfs/volumes/Raid5-4/ex/ /vmfs/volumes/Raid5-4/exnew*

This new machine can then be registered by browsing to vmx file, right click, register virtual machine, when turning on do as recommended and create new ID.

Step 2 (the easy way, most people report this to work, but not always...)

Create new snapshot on machine

Try to apply all snapshots (delete all) in VC

Step 3 (using the command line)

Run following from a putty session (alter path to correct VMX)

vmware-cmd vmfs/volume/Raid5-4/ex/ex.vmx removesnapshots

This should remove all snapshots

This takes can take ALONG time, but do not presume it doing nothing, and do not stop once started. You can check what it's doing using WINscp (or anther method of checking if file size is changing, I do not trust browsing the data stores) Keep refreshing and you should see files sizes altering).

Alternative step (some report below works, but did not for me)

As an alternative to above people with similar problems have reported fixing it by using VMware tools to clone the disk, but this did not work for me.

vmkfstools -i /vmfs/volumes/Raid5-4/newex/ex_2.vmdk /vmfs/volumes/Raid5-4/newex/ex_2new.vmdk

Then re-point disk to new.vmdk (NOTE once you have re-pointed disk to new one it's VERY tricky to point back to snapshooted version, only do this if have a good backup!

Other useful command if a host stops responding, or fails to complete a command due to file in use type error (but be carful not to do this while snapshots are being removed):-

service mgmt-vmware restart

Also consider consequences if bring up a back up machine that you may decide not to keep, e.g. I disabled mailsweeper delivery service before started.

View solution in original post

Reply
0 Kudos
5 Replies
demz
Expert
Expert
Jump to solution

In the COS you can try :

vmware-cmd /path/to/your/VM.vmx removesnapshots

You can also look at the coherence between the parent ID in the snapshot VMDK and the original VMDK.

Look at the content of the .vmsd file too.

Pizzaman
Contributor
Contributor
Jump to solution

We have had similar problems here in the past, below is a document I created for internal use if it happened while I am not here.

Forced snapshoot removal.

Step 1 (backup, or can use anther method you are confident of restoring from)

Turn off machine

Make a backup of machine, this can be done with following command from a putty session (alter path)

cp -r /vmfs/volumes/Raid5-4/ex/ /vmfs/volumes/Raid5-4/exnew*

This new machine can then be registered by browsing to vmx file, right click, register virtual machine, when turning on do as recommended and create new ID.

Step 2 (the easy way, most people report this to work, but not always...)

Create new snapshot on machine

Try to apply all snapshots (delete all) in VC

Step 3 (using the command line)

Run following from a putty session (alter path to correct VMX)

vmware-cmd vmfs/volume/Raid5-4/ex/ex.vmx removesnapshots

This should remove all snapshots

This takes can take ALONG time, but do not presume it doing nothing, and do not stop once started. You can check what it's doing using WINscp (or anther method of checking if file size is changing, I do not trust browsing the data stores) Keep refreshing and you should see files sizes altering).

Alternative step (some report below works, but did not for me)

As an alternative to above people with similar problems have reported fixing it by using VMware tools to clone the disk, but this did not work for me.

vmkfstools -i /vmfs/volumes/Raid5-4/newex/ex_2.vmdk /vmfs/volumes/Raid5-4/newex/ex_2new.vmdk

Then re-point disk to new.vmdk (NOTE once you have re-pointed disk to new one it's VERY tricky to point back to snapshooted version, only do this if have a good backup!

Other useful command if a host stops responding, or fails to complete a command due to file in use type error (but be carful not to do this while snapshots are being removed):-

service mgmt-vmware restart

Also consider consequences if bring up a back up machine that you may decide not to keep, e.g. I disabled mailsweeper delivery service before started.

Reply
0 Kudos
Kosch
Contributor
Contributor
Jump to solution

Thanks for the replies chaps I'll give this a go later this afternoon and then dish out some points Smiley Happy

Cheers

Doug

Reply
0 Kudos
Pizzaman
Contributor
Contributor
Jump to solution

Hope it goes well.

Just looked other the document and it's not clear that the machine should be turned off before you try and remove the snapshots in step 2 and 3.

Reply
0 Kudos
Kosch
Contributor
Contributor
Jump to solution

Taking another set of snapshots for Server A (Dev) and then deleting them worked like a treat.

Havent had a chance to test Server B (Live)

Reply
0 Kudos