Unremovable invisible snapshot

[daveli]

We are running ESX 3.5.0 and are using VCB for making backups. Yesterday VCB failed to remove its snapshot. That has happened a couple of times before, and I removed it manually as usual. But this time the VCB snapshot got replaced by a "Consolidate helper" snapshot instead.

We have had a snapshot that was hard to delete before, so I knew that if I created another snapshot and then selected "Delete all" i Snapshot Manager, it should delete all snapshots. So at the end of the day I shutdown the server, created another snapshot and then deleted all snapshots. All snapshots disappeared from Snapshot Manager, and I went home.

Today the server had been "paused" by VMware, since all disk space on the LUN had been filled by snapshot delta/redo/vmdk files. I had to move another servers disk from the LUN to make it resume again. So the snapshot was not removed after all. There are now several delta/redo/vmdk files on the LUN, but no snapshot i the Snapshot Manager in the GUI.

How do I remove this snapshot?

[Chuck8773]

Welcome to the forums!! I hope you find the answers that you are looking for.

To start, this is a good one. No snapshots on one vmdk. 5 on two of the others. and 15 on the fourth. Looks like the process started taking snapshots around 8:20 pm on Wednesday. Is this when the VCB backup started? You mentioned that you already tried to create a snapshot and "delete all". Can you try that with the VM powered off?

First though, make sure you have a way to get back to this point. Take snapshots of the LUN, or copy the files somewhere.

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

[daveli]

Thank you!

We have had some problems with the scheduled backup this week, so I started a manual backup during the day on wednesday. But yes, 8:20 pm should be the time that VCB created its snapshot.

I did try to remove the snapshot more than once, and before my last try I did power of the VM first.

I actually had the exact same problem in the beginning of this week. But then it was a internal webserver, where no files had been updated since the snapshop was created, so I just deleted the delta files manually and edited the vmx file. But this server is a databaser server used by several of our customers, so I can't shut it down during the day.

Is there another way to merge delta files with the original vmdk files, other than via the GUI?

[Chuck8773]

With something as critical as this, I would open a case with VMWare. They should have the tools and ability to fix this for you. Do you have a support contract?

It would be a good thing to know how to fix this on your own, but on live customer data is not the appropriate place to learn.

Charles Killmer, VCP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

[avlieshout · Accepted solution]

This sure looks like a complicated one.

I would avoid any risk since it's a production server.

I recommend using VMware Converter to clone this vm to a new machine. During the conversion all snapshots get consolidated.

-Arnim van Lieshout

-

---

Blogging: http://www.van-lieshout.com

Twitter:

If you find this information useful, please award points for "correct" or "helpful".

[daveli]

Thank you. I'll try creating a clone later today, when I can shut down the server.

-David Eliason

[azn2kew]

You can also use System State backup of the virtual machine and archive it in case you've damage it. It can be restore to a new virtual machine anytime which makes life easier as well. Or using P2V converter to clone a new version of it so that all snapshots are committed and cleared and delete the old server. Have you try using RVTools, it will help you delete snaphosts as well and its pretty good ESX admin tool kit.

If you found this information useful, please consider awarding points for "Correct" or "Helpful". Thanks!!!

Regards,

Stefan Nguyen

VMware vExpert 2009

iGeek Systems Inc.

VMware, Citrix, Microsoft Consultant

[daveli]

It took a while to clone the machine, but it worked perfectly. Thank you for your help.

-David Eliason