Created a snapshot manually, now they continue to fill up storage; error quiescing

[CaptainH]

A few days ago I manually created a snapshot in vSphere 5.1 on my Exchange server.  Shortly thereafter, my VM shutdown due to lack of storage space.  I deleted the VMDKs with 0001, 0002, etc. and this caused issues (obviously).  So I restored a snapshot from my SAN on the volume level and was able to consolidate the snapshots and free up enough space to start the VM.  All was running well for a few days, when I suddenly started receiving this error again and the VM shut down.  Over the last several days this snapshot is repeatedly being recreated somehow, even though I don't have anything scheduled (I can't even find WHERE to schedule this!) and it is taking my entire Exchange server down.  I start receiving an error stating that VMware is unable to quiesce and then shortly after the VM shuts down again due to lack of space.  I've already given it more space from my SAN, but this still continues to happen.

Each time I am able to simply consolidate the snapshots through VMware, but it has gone down three times in the last twenty four hours.  How do I stop these snapshots??  Help!!!!

[jrmunday]

Have a look at this KB article;

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=200363...

and this one (from the link above);

http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=&popup=true&languageId=&externalID=1...

[continuum · Accepted solution]

If you do not create snapshots yourself they are either created by a (misbehaving) automatic backup tool or by Autoprotect.
Autoprotect is rarely used on ESXi so what about your backup-tools ?

[CaptainH]

I believe this is an issue with my SAN's snapshots.  It does integrate with VMware and it seems that it is leaving the snapshots behind.  The only other backup tools I have are Backup Exec 12.5 and the built-in Windows backup and this issue began before they were in place on this particular server.

The real issue I have is that I turned off the snapshot integration in the SAN on the OS volume that was having this trouble and 2 hours later, it still ran out of space and I had to manually consolidate the snapshots.

I just spoke to my SAN vendor and they suggested another method of quiescing the Exchange databases as opposed to the VMware integration in their product.  I have turned off VSS integration entirely on the SAN and I hope this fixed the problem.  I am still am not fully confident that this is going to solve the problem because I turned off snapshots entirely on the volume in question and 2 hours later, it still ocurred.

[a_p_]

From what you say I think this could be an issue with a snapshot that's not displayed in the Snapshot Manager!? If your virtual disks have been thick provisioned and there are any ..00000x.vmdk files in the VM's folder on the datastore, you may create a new Snapshot ant then use the "Delete All" button to merge all existing snapshots - regardless of whether they are shown in the Snapshot Manager - into the base disk. Depending on the free disk space on the datastore, you should consider to do this with the VM powered off.

If you are unsure, please post a screen shot of the datastore browser window showing all the VM's files with all details like sizes, time stamps, ...

André

[CaptainH]

You were correct, it was my SAN that was continuing to create snapshots and not deleting the previous snapshots out of the volume, thus filling it up.  It would work fine for awhile and then all of a sudden get tripped up and then stop deleting.  As I moved more mailboxes to the server, the snapshots became larger, thus filling it up faster.  This is a "bug" as they're calling it with the newest software on my SAN.