12 Replies Latest reply on Jan 23, 2009 1:56 PM by jftwp

    No snapshot but we have  big delta files

    jfsardon Novice

      We have sereral 3.0.1 ESX serveurs

      We have created snapshot on 2 VMs with 500 Go virtual RDM  ==> OK

       

      two weeks after we destroy the snapshots, but :

      \- the delta file of the RDM are still on the vmfs volume, are still growing

      \- no snapshot in the snapshot manager

      \- use vmware-cmd hassnapshots[/b] = 0

       

      now we have a 50 Go delta file !

       

      How can I merge the delta file with the raw device or do I copy the data onto another volume and destroy the primary RDM ??

        • 1. Re: No snapshot but we have  big delta files
          jfsardon Novice

          note We are not using vcb or another backup using the ESX's snapshot...

          • 2. Re: No snapshot but we have  big delta files
            cheeko Master

            just on thought: is that VM in undoable mode? if so you'll be promted to merge the delta to your VMDK when you boot up your VM.

            • 3. Re: No snapshot but we have  big delta files
              asp24 Enthusiast

              any solution?

              I'm having a VM with a big delta file, but snapshot manager is not showing any. I think a backup script did this (crashed)

               

              Is there a way to force the files to be committed?

              If I try to create a new snapshot I get a "invalid snapshot configuration..."

              • 4. Re: No snapshot but we have  big delta files
                Champion

                Sounds like the VC snapshot database has got out of sync with the VM. Remove and re-register the VM from the VC inventory and it should pick up the snapshots and then you can remoev them through the snapshot manager. (NB: do not 'remove from disk', remove from inventory!)

                • 5. Re: No snapshot but we have  big delta files
                  saschmidt Enthusiast

                  Hi,

                   

                  I've got this kind of problem, too... I'm having 2 delta files and no snapshot is display... Neither within the VC nor with "vmware-cmd /vmfs/volumes/.../ hassnapshot"...

                   

                  I'd like to commit the existing delta-files but don't know how...

                   

                  Any suggestions?

                   

                  with kinds regards,

                  Sascha

                  • 6. Re: No snapshot but we have  big delta files
                    Champion

                    I had this problem with one of my VMs, unfortunately I could not get the delta's commited to had to go through the tedious process of creating a new VM and imaging the data from old->new to get out of the situation. This was the only resolution VMware support could suggest!

                     

                    Snapshots are flakey, and the larger the delta (i.e. the busier the VM and the longer the snapshot is left in place), the greater the chance you'll have problems comitting them.

                     

                    You can try un-registering and re-registering the VM with VC, this sometimes causes the snapshot manager to pick up the 'missing' snapshots, if they are still listed in the .vmsd/.vmsn files. It's a safe process so worth a go before trying anything else.

                     

                    Check that your VM does not suffer from this "problem". If it does, do not try adding anymore snapshots.

                     

                    http://kb.vmware.com/kb/5096672

                     

                    Otherwise you can try the following - point the VI client directly at the ESX host running the VM, rather than VC as it has a hard-coded timeout which is too short. Power off the VM, create a new snapshot, and then try and commit. I've been told by support that this \*should* clear the stuck delta's, but in my case it didn't work.

                     

                    I also recommend you take a full backup of the VM before trying this.

                     

                    Good luck, I've been there and I don't envy your situation.

                    • 7. Re: No snapshot but we have  big delta files
                      andr3wc Novice

                      Well I've just had this battle today.

                      I didn't know how snapshots work, so no idea my Delta-s were silently growing, and growing and growing.

                      I wanted to export a VM to clone it at another site, and the VMimporter failed.

                      I looked at the disk and OMFG its 18 GB (machine is 5 GB).

                      To get this machine working I had taken 6 snapshots as we tried different things, rolled back, tried something else etc (snapshot feature is great).

                      Once working, I left it to work.

                       

                      So today I click the "delete all" button and it races to 95% complete and then "hangs".  Watching "top" in the console shows no activity.  However, the timestamps on the delta's keep changing.  So I hope.

                       

                      Then it goes NUTS.  Consuming several hundred MB of disk every few seconds.  I frantically delete shit (ISO's mostly) and blow away trashy VMs in the hope of enabling enough disk space.

                       

                      I fail.

                       

                      Disk full error message.  VM now consuming 25 GB of disk with monster delta files, but VIC says "no snapshots".  %$@#$

                      I do the unregister/register thing, but no joy.

                      I create a new snapshot, and then do the "delete all" from the command line.

                      Hmmm, taking a long time for a snapshot that should contain zero changes (machine was down all this time).

                       

                      Again, the disk munching resumes.  I think about two other snapshots that exist for all the other machines, and decide I hate snapshots and delete them from the command line.

                       

                      Now it could be a coincidence, but when running one of these other commands in another putty session, my main delete fails spetactularly with:

                       

                      VMControl error -999: Unknown error: SoapError: ServerFaultCode(1514) : (The request refers to an object that no longer exists or has never existed.)

                       

                      I swear, but my foul language is not required. The deltas are gone.  Disk space is back.  I open VIC, and hit boot, and pray.  It boots.  Evenvwr shows it is not some weird rolled back thing from May, but was running today.

                       

                      So, there you go.  Just verbose confirmation of the messages above.

                      Shutdown

                      Unregister.

                      Register.

                      Take a snapshot

                      Ensure you have lots of disk space, and LOTS of time (takes hours).

                      Delete all snapshots.

                      Pray

                      • 8. Re: No snapshot but we have  big delta files
                        SFMarkham Novice
                        mittell wrote:

                        Otherwise you can try the following - point the VI client directly at the ESX host running the VM, rather than VC as it has a hard-coded timeout which is too short. Power off the VM, create a new snapshot, and then try and commit. I've been told by support that this \*should* clear the stuck delta's, but in my case it didn't work.

                         

                        I also recommend you take a full backup of the VM before trying this.

                         

                        Just an FYI this DID work for me with a stuck snapshot that was 500MB in size.

                         

                        Thank you for posting this

                        Shawn

                        • 9. Re: No snapshot but we have  big delta files
                          maccajuk Novice

                          Hi, just a quick note based on my experience from today (whilst it is still fresh in my mind). As per suggestions above we did the following.

                           

                          We actually connected through VI to the VC box. (I seemed to have skipped that part of Mittel's suggestion?!?!)

                           

                          1. Shutdown the VM

                          2. Take a new snapshot in Snapshot manager

                          3. Delete ALL snapshots in Snapshot manager

                          4. Restart server

                           

                          It's worth mentioning that on several occasions between actions 3 and 4, I thought it had failed. Including the 'Operation timed out' error. However, looking on the ESX server we could see the delta files were still being modified. We had to keep an eye on space and actually delete an old VM on the go, but eventually, about an hour later, it completed. All delta files had gone and we were left with one shiny vmdk.

                           

                          Server started happily with events in the Windows system log from the last time the box was up.

                          • 10. Re: No snapshot but we have  big delta files
                            osde.info Expert

                             

                            apparently there is a bug in VMWVI that it jumps to 95% complete and always times out after 15mins (even if it isn't)

                             

                             

                            the only workaround is to manually watch the delta files (in the way you did)

                             

                             

                            if this is a problem for you and if you've got support you might like to log an SR !

                             

                             

                             

                             

                             

                            regards

                            clive (aka osde.info)

                            http://www.vizzr.info

                             

                            points=prizes

                            so please click helpful answer or correct answer when appropriate

                            (but where's the unhelpful button ?)

                             

                             

                            • 11. Re: No snapshot but we have  big delta files
                              DonCarter Lurker

                              If you connect to the host that the particular VM resides on, you can see the progress...even though in VC it says that the "Operation Time Out".

                              • 12. Re: No snapshot but we have  big delta files
                                jftwp Expert

                                 

                                Above suggestions did the trick for me; was in same situation as others -


                                snapshot manager showing nothing, but VM had snapshot/delta files that were growing.

                                 

                                 

                                Slightly modified as follows:

                                 

                                 

                                As a contingency, first take a backup or even 'clone' the existing VM to another datastore or whatever.  Something to fall back on.

                                 

                                 

                                1. Shutdown the VM.

                                2. Take a new snapshot in Snapshot manager, directly connected to the ESX host the VM is currently on (to work around VirtualCenter 15 minute timeout issue discussed above)

                                3. Delete ALL snapshots in Snapshot manager  (even if there's just the one listed that you just took, use the 'Delete All' button).  Monitor progress... your time to finish 'may vary' depending on snapshot size/s.

                                4. Once VIC confirms snapshot deletions are completed, check VMs datastore files either via GUI or CLI.  Should see no 'delta' files anymore.  Restart server.