VMware

This Question is Answered

1 2 Previous Next 20 Replies Last post: Apr 14, 2009 1:58 PM by cxo  

Deleting snapshots hangs VM? posted: Feb 19, 2009 4:03 PM

Click to view mattjk's profile Enthusiast 80 posts since
Dec 19, 2007
Hi all,

Whenever we delete a VM snapshot via VIC, it always seems to make the VM's guest OS pause a few times during the process.

The size of the snapshot seems to affect the number / length of these pauses, but as an example I just deleted a ~500MB snapshot on a VM with 4 disks and it caused the VM to pause 3-4 times for 5-10 seconds at a time. The pauses seem to occur no matter what the guest OS is - W2K3, W2K8, 32 or 64-bit. All VMs have the VMWare Tools installed, and we don't have any particular resource constraints.

Is this normal? Is there anyway to prevent these pauses?

Cheers,
Matt

Re: Deleting snapshots hangs VM?

1. Feb 19, 2009 4:34 PM in response to: mattjk
Click to view Lightbulb's profile Virtuoso 1,391 posts since
Aug 15, 2008

Local Storage or SAN?

Check your /var/log/vmkernel log for SCSI errors around the time when you delete the snapshots.

Re: Deleting snapshots hangs VM?

3. Feb 19, 2009 5:01 PM in response to: mattjk
Click to view Lightbulb's profile Virtuoso 1,391 posts since
Aug 15, 2008

Well you are not alone

http://communities.vmware.com/thread/188876

I think there is a good chance the erros in vmkernel are related to your issue. As to cause hmm don't have that yet.

Might want to look at this

http://virtrix.blogspot.com/2007/06/vmware-dreadful-sticky-snapshot.html


Re: Deleting snapshots hangs VM?

5. Feb 20, 2009 12:32 AM in response to: mattjk
Click to view Lightbulb's profile Virtuoso 1,391 posts since
Aug 15, 2008
Let us know how it turns out. Good luck and sorry I could not be more help.

Re: Deleting snapshots hangs VM?

6. Feb 20, 2009 10:52 AM in response to: mattjk
Click to view glynnd1's profile Expert 615 posts since
Dec 3, 2004
Matt, what kind of disks do you have in you NetApp?
I saw this at a previous job where we have a FAS3020 with SATA, though it was limited to VMs that were doing a decent amount of disk writes during the snapshot period.
We found that while the snapshot was been merged in a second delta file would be create to handle current writes and during this time we saw no issues, but when this second delta file was been merged in we'd see the VM enter a paused state from internal or external monitoring. Moving these VMs to FC disk resolved the problem as the snapshots were smaller - faster VCB backup, the resulting delta file created during merge was smaller and the merge of the delta did not cause a noticeable interruption to the OS or application.

Re: Deleting snapshots hangs VM?

8. Feb 21, 2009 12:21 PM in response to: mattjk
Click to view titaniumlegs's profile Enthusiast 52 posts since
Feb 28, 2005
You probably need to apply and activate patch ESX350-200808401-BG. You said you have ESX3.5u3, so the patch is built in. You activate it by inserting
prefvmx.consolidateDeleteNFSLocks = "TRUE"
in /etc/vmware/config and reboot ESX. (VMotion, shutdown or suspend VMs first)
Details in http://media.netapp.com/documents/tr-3428.pdf p.12-13.

Enjoy!

Re: Deleting snapshots hangs VM?

10. Feb 22, 2009 5:48 PM in response to: mattjk
Click to view titaniumlegs's profile Enthusiast 52 posts since
Feb 28, 2005

TR3428 gets updates every 2-3 months because it covers a lot of material, and some of it changes or we find new information to include.

AFAIK, we've never seen or heard of data loss as a result of not having the patch - just the long VM hangs during snap commit.

I can't answer for VMware's tech support, but thanks for the props on ours!

CYa

Peter

Re: Deleting snapshots hangs VM?

12. Feb 28, 2009 7:32 PM in response to: mattjk
Click to view RKCCruiser's profile Novice 2 posts since
Feb 28, 2009
Hello Peter,


Thanks for the information. We are having the same problems with a setup. We are also using NFS with two ESX 3.5u3 servers. We are using a product by Vizioncore that basically does a lot of underlying scripting for snapshotting from one storage box to another. We have the same issue whenever it states that it is "deleting snapshots". We applied the fix as outlined in the documentation you recommended and rebooted. But whenever our snapshot process completes and deletes the old snapshot, the VM is hung up for about 1-2 minutes and no one can access that VM from the network. Any ideas?

Re: Deleting snapshots hangs VM?

13. Mar 1, 2009 11:09 PM in response to: RKCCruiser
Click to view titaniumlegs's profile Enthusiast 52 posts since
Feb 28, 2005

Hi!

It sounds like the patch isn't activated. The 1-2 minute hang on VM snapshot delete is the basic symptom.

Check to make sure the quotes around true are double quotes, there's a space on each side of the = (not sure that matters, but that's how I have it and the rest of the options in that file), and no other quotes.

We had some problems with TR3428 and Word "helping" us by converting quotes to "smart quotes" and dashes to long dashes, so when you cut+paste, it wasn't always what you expected.

Post or PM me a copy of /etc/vmware/config and I'll take a look if you like.

Peter

Re: Deleting snapshots hangs VM?

14. Mar 2, 2009 6:32 AM in response to: titaniumlegs
Click to view RKCCruiser's profile Novice 2 posts since
Feb 28, 2009
Thank you, Peter.

My co-worker who has been working with me discovered that we had only installed the Update 2 for this customer and we upgraded their ESX servers to Update 3 yesterday and all is resolved. We now have the snapshot deletions taking place with only momentary connection loss for about 1 ping, which should be more than acceptable for our apps. Thank you again so much for the information.

James

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities