VMware Cloud Community
jfsardon
Contributor
Contributor

No snapshot but we have big delta files

We have sereral 3.0.1 ESX serveurs

We have created snapshot on 2 VMs with 500 Go virtual RDM ==> OK

two weeks after we destroy the snapshots, but :

\- the delta file of the RDM are still on the vmfs volume, are still growing

\- no snapshot in the snapshot manager

\- use vmware-cmd hassnapshots[/b] = 0

now we have a 50 Go delta file !

How can I merge the delta file with the raw device or do I copy the data onto another volume and destroy the primary RDM ??

12 Replies
jfsardon
Contributor
Contributor

note We are not using vcb or another backup using the ESX's snapshot...

Reply
0 Kudos
cheeko
Expert
Expert

just on thought: is that VM in undoable mode? if so you'll be promted to merge the delta to your VMDK when you boot up your VM.

Reply
0 Kudos
asp24
Enthusiast
Enthusiast

any solution?

I'm having a VM with a big delta file, but snapshot manager is not showing any. I think a backup script did this (crashed)

Is there a way to force the files to be committed?

If I try to create a new snapshot I get a "invalid snapshot configuration..."

Reply
0 Kudos
admin
Immortal
Immortal

Sounds like the VC snapshot database has got out of sync with the VM. Remove and re-register the VM from the VC inventory and it should pick up the snapshots and then you can remoev them through the snapshot manager. (NB: do not 'remove from disk', remove from inventory!)

Reply
0 Kudos
saschmidt
Contributor
Contributor

Hi,

I've got this kind of problem, too... I'm having 2 delta files and no snapshot is display... Neither within the VC nor with "vmware-cmd /vmfs/volumes/.../ hassnapshot"...

I'd like to commit the existing delta-files but don't know how...

Any suggestions?

with kinds regards,

Sascha

Reply
0 Kudos
admin
Immortal
Immortal

I had this problem with one of my VMs, unfortunately I could not get the delta's commited to had to go through the tedious process of creating a new VM and imaging the data from old->new to get out of the situation. This was the only resolution VMware support could suggest!

Snapshots are flakey, and the larger the delta (i.e. the busier the VM and the longer the snapshot is left in place), the greater the chance you'll have problems comitting them.

You can try un-registering and re-registering the VM with VC, this sometimes causes the snapshot manager to pick up the 'missing' snapshots, if they are still listed in the .vmsd/.vmsn files. It's a safe process so worth a go before trying anything else.

Check that your VM does not suffer from this "problem". If it does, do not try adding anymore snapshots.

http://kb.vmware.com/kb/5096672

Otherwise you can try the following - point the VI client directly at the ESX host running the VM, rather than VC as it has a hard-coded timeout which is too short. Power off the VM, create a new snapshot, and then try and commit. I've been told by support that this \*should* clear the stuck delta's, but in my case it didn't work.

I also recommend you take a full backup of the VM before trying this.

Good luck, I've been there and I don't envy your situation.

Reply
0 Kudos
andr3wc
Contributor
Contributor

Well I've just had this battle today.

I didn't know how snapshots work, so no idea my Delta-s were silently growing, and growing and growing.

I wanted to export a VM to clone it at another site, and the VMimporter failed.

I looked at the disk and OMFG its 18 GB (machine is 5 GB).

To get this machine working I had taken 6 snapshots as we tried different things, rolled back, tried something else etc (snapshot feature is great).

Once working, I left it to work.

So today I click the "delete all" button and it races to 95% complete and then "hangs". Watching "top" in the console shows no activity. However, the timestamps on the delta's keep changing. So I hope.

Then it goes NUTS. Consuming several hundred MB of disk every few seconds. I frantically delete shit (ISO's mostly) and blow away trashy VMs in the hope of enabling enough disk space.

I fail.

Disk full error message. VM now consuming 25 GB of disk with monster delta files, but VIC says "no snapshots". %$@#$

I do the unregister/register thing, but no joy.

I create a new snapshot, and then do the "delete all" from the command line.

Hmmm, taking a long time for a snapshot that should contain zero changes (machine was down all this time).

Again, the disk munching resumes. I think about two other snapshots that exist for all the other machines, and decide I hate snapshots and delete them from the command line.

Now it could be a coincidence, but when running one of these other commands in another putty session, my main delete fails spetactularly with:

VMControl error -999: Unknown error: SoapError: ServerFaultCode(1514) : (The request refers to an object that no longer exists or has never existed.)

I swear, but my foul language is not required. The deltas are gone. Disk space is back. I open VIC, and hit boot, and pray. It boots. Evenvwr shows it is not some weird rolled back thing from May, but was running today.

So, there you go. Just verbose confirmation of the messages above.

Shutdown

Unregister.

Register.

Take a snapshot

Ensure you have lots of disk space, and LOTS of time (takes hours).

Delete all snapshots.

Pray

Reply
0 Kudos
SFMarkham
Contributor
Contributor

Otherwise you can try the following - point the VI client directly at the ESX host running the VM, rather than VC as it has a hard-coded timeout which is too short. Power off the VM, create a new snapshot, and then try and commit. I've been told by support that this \*should* clear the stuck delta's, but in my case it didn't work.

I also recommend you take a full backup of the VM before trying this.

Just an FYI this DID work for me with a stuck snapshot that was 500MB in size.

Thank you for posting this

Shawn

Reply
0 Kudos
maccajuk
Enthusiast
Enthusiast

Hi, just a quick note based on my experience from today (whilst it is still fresh in my mind). As per suggestions above we did the following.

We actually connected through VI to the VC box. (I seemed to have skipped that part of Mittel's suggestion?!?!)

1. Shutdown the VM

2. Take a new snapshot in Snapshot manager

3. Delete ALL snapshots in Snapshot manager

4. Restart server

It's worth mentioning that on several occasions between actions 3 and 4, I thought it had failed. Including the 'Operation timed out' error. However, looking on the ESX server we could see the delta files were still being modified. We had to keep an eye on space and actually delete an old VM on the go, but eventually, about an hour later, it completed. All delta files had gone and we were left with one shiny vmdk.

Server started happily with events in the Windows system log from the last time the box was up.

Reply
0 Kudos
osde_info
Enthusiast
Enthusiast

apparently there is a bug in VMWVI that it jumps to 95% complete and always times out after 15mins (even if it isn't)

the only workaround is to manually watch the delta files (in the way you did)

if this is a problem for you and if you've got support you might like to log an SR !

regards

clive (aka osde.info)

points=prizes

so please click helpful answer or correct answer when appropriate

(but where's the unhelpful button ?)

regards clive http://vizz.info
Reply
0 Kudos
DonCarter
Contributor
Contributor

If you connect to the host that the particular VM resides on, you can see the progress...even though in VC it says that the "Operation Time Out".

Reply
0 Kudos
jftwp
Enthusiast
Enthusiast

Above suggestions did the trick for me; was in same situation as others -


snapshot manager showing nothing, but VM had snapshot/delta files that were growing.

Slightly modified as follows:

As a contingency, first take a backup or even 'clone' the existing VM to another datastore or whatever. Something to fall back on.

1. Shutdown the VM.

2. Take a new snapshot in Snapshot manager, directly connected to the ESX host the VM is currently on (to work around VirtualCenter 15 minute timeout issue discussed above)

3. Delete ALL snapshots in Snapshot manager (even if there's just the one listed that you just took, use the 'Delete All' button). Monitor progress... your time to finish 'may vary' depending on snapshot size/s.

4. Once VIC confirms snapshot deletions are completed, check VMs datastore files either via GUI or CLI. Should see no 'delta' files anymore. Restart server.