VMware Cloud Community
kot99
Contributor
Contributor

Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine.

Then i try to backup VM using VDR the following error occured:

Create virtual machine snapshot

Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine.

How can i fix this issue ?

Please help...thanks.

Reply
0 Kudos
37 Replies
tauberc
Contributor
Contributor

i had the same problem with two windows 2003 webservers.

I ran the Security Configuration Wizard on both servers and the wizard disabled the services "Microsoft Software Shadow Copy Provider" and "Volume Shadow Copy"

i set the startup of both services from "disabled" to "manual" and snapshots are working fine now.

also ensure that the "VMware Shadow Copy" service is set to manual startup

hope this helps

Reply
0 Kudos
LANanni
Contributor
Contributor

Hey- Just wondering if this error was constant or more intermittent; as in it happens when multiple snapshots in succession as part of a backup routine? im running Backup Exec 12.5 with VCB and no integration module (thank GOD!) ive found if i manually run the backup job that intern requests the snapshot be created, its fine.

Also im kinda new to the VM world especially in regards to backing guests up with VCB.... Where can i find the Security Configuration Wizard you mentioned? thanks for any/all help.

Cheers,

LA-

Reply
0 Kudos
tauberc
Contributor
Contributor

the error was constant, backup did never work.

I'm using the VMware Data Recovery Appliance for backup.

the Security Configuration Wizard is part of windows 2003 you can add/remove it through the windows components

but if you didn't use it your system was messed by something else.

If you do a manual snapshot via vsphere client be sure to uncheck "snapshot the virtual machine's memory" and check "quiesce guest file system(need vmware tools installed)

this also produces the same error a the backup appliance if the two mentioned services are disabled

Reply
0 Kudos
MitchHP
Enthusiast
Enthusiast

I am facing the same problem.

I already checked those two services but both are set to manual.

Each time the backup tries to start it fails with the mentioned failure "Cannot create a quiesced snapshot because the create snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine.".

Any other ideas?

Reply
0 Kudos
robc_yk
Enthusiast
Enthusiast

We get the same error here on a few servers.

If I make some headway I will report, if someone else could do the same that would be great.

Cheers.

Reply
0 Kudos
dconvery
Champion
Champion

First off, make sure the sync driver and VSS providers have been installed with VMware Tools. They did NOT get installed on some previous versions, so they may be missing if you upgraded.

Also, I see this a lot with busy transactional systems, like SQL, Exchange and Domain Controllers. In my opinion (and a few other people here) you should never use a snapshot based backup for these. You should use agent based backups. To get an image for fast recovery, uninstall the sync and VSS from VMWare Tools and take a crash consistent backup. Recovery would be to restore the VM from VDR/VCB and then use the agent to recover the transactional data.

Dave Convery

VMware vExpert 2009

Careful. We don't want to learn from this.

Bill Watterson, "Calvin and Hobbes"

Dave Convery, VCDX-DCV #20 ** http://www.tech-tap.com ** http://twitter.com/dconvery ** "Careful. We don't want to learn from this." -Bill Watterson, "Calvin and Hobbes"
Reply
0 Kudos
thecakeisalie
Enthusiast
Enthusiast

The VSS component is what is causing the problem you're facing, and there are a lot of different ways to workaround it, but the main thing you need to remember is that if you're not running VSS during your backup operations, you will not have an application or filesystem consistent backup, you will only have a crash consistent backup.

The main reason for VSS not working as expected is because the tools aren't installed properly, but sometimes it's just a matter of the VSS in Windows not being able to quiesce the filesystem because an application is demanding access from it, so it can't 'pause' the I/O within the VM (this commonly occurs in SQL/Exchange/Oracle/Citrix), or the VSS just isn't functioning correctly within the guest so when the VMware tools attempt to use the VSS to quiesce, it fails with an error that you can see in the Event logs.

The following KB's should help you diagnose the problem, and the first one -although the actual situation might not apply to you directly- the information within it will tell you how to disable the VSS/SYNC driver within the VM, the others help identify if the VSS is working within the guest (using ntbackup), and provide workarounds to known MS issues that prevent VSS from running correctly. Remember also that you can only use SYNC or the VSS component, so when you're running the VMware tools installation, make sure you do it interactively so you can see which option is selected.

Unable to take a quiesced VMware snapshot of a virtual machine - http://kb.vmware.com/kb/1009073

Troubleshooting Volume Shadow Copy quiesce related issues - http://kb.vmware.com/kb/1007696

64bit Windows virtual machines generate errors when trying to use the VSS and NT backup - http://kb.vmware.com/kb/1003383

Reply
0 Kudos
RParker
Immortal
Immortal

Confirmed Synch Driver in device manager was the problem, although you don't need a restart, just stop the service and set it to disable, which has to be done in two steps....

Reply
0 Kudos
RUG201110141
Enthusiast
Enthusiast

Here is a KB from Microsoft that explained the issue for me -

Reply
0 Kudos
AlbertWT
Virtuoso
Virtuoso

To All,

The

following tricks working without uninstalling the VMWare tools support

for VSS. Please be advised that I've tried this method on all of my failed to backup of Windows Server 2008/2003 x64 VMs

The reason why I'm doing this again is that I feel uncomfortable in removing the VSS snapshot provider support in VMWare tools:

1. Backup the following regkey: _HKLM\Software\Microsoft\EventSystem\ _

2. Erase the regkey HKLM\Software\Microsoft\EventSystem\{26c409cc-ae86-11d1-b616-00805fc79216} an below

3. Reboot.

4. Verify that things are working by running

vssadmin list writers

vssadmin list providers.

and then in the services.msc list of services snap ins, make sure the following services are Started Automatically:

1. COM+ System Application

2. Distributed Transaction Coordinator

3. Volume Shadow Copy

4. VMware Physical Disk Helper Service

Automatically but not started:

1. Microsoft Software Shadow Copy Provider

2. VMware Snapshot Provider

now the backup is all working great Smiley Happy

This is the solution that i tried around myself without uninstalling the VSS Provider support in VMWare tools.

See the following KB URL: http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1009073&sl...KB_1_1&dialogID=38752513&stateId=0%200%2038754922_

Hope this can be a helpful post for all of you.

Kind Regards,

AWT

/* Please feel free to provide any comments or input you may have. */
Reply
0 Kudos
tprevot
Contributor
Contributor

If the service " VMWARE Snapshot Provider" don't appear in the list of windows services, we have the same error (-3960).

It cause Snapshot with "Quiesce" option and backup with Data Recovery don't work.

In my case, I have followed the procedure of AlbertWT, and desinstall the WMWARE Tools.

I 've setup on automatic startup the service "Windows Management Instrumentation". Reboot. Install again the VMWare Tools.

The Service VMWARE Snapshot Provider is now present in the list of services, and start automatically.

The Backup w/ Datarecovery on this VM works perfect.

Reply
0 Kudos
zzAngelo
Contributor
Contributor

I have a similar problem with one vm in my infrastructure. I have done a disk defrag, removed and reinstalled the vmtools, followed the steps suggested by AlbertWT, but the error is still there....

the vm is a win 2003 cold imported from a IBM server. I have another vm cold imported from another IBM server that works fine. The only difference is that the failing vm is a Citrix server with a SQL express installed.

Any other suggestion?

Thanks in advance

Reply
0 Kudos
mforbes
Enthusiast
Enthusiast

Any updates on this at all? I have a similar situation, only I CAN take snapshots manually with no issues at all. When I try to run vDR backups manually it fails immediately.

Here is the error message I get: Failed to create snapshot for XXX-XX-WS, error -3960 ( cannot quiesce virtual machine)

6/3/2010 10:36:54 AM: Task incomplete

I am on ESXi 4.0 (update 1 patched to 256968) running View 4.01 on a 2008 R2 server that is not backing up.

Mike Forbes

Mike Forbes
Reply
0 Kudos
clevelas
Enthusiast
Enthusiast

I was running into the same problem today. My guess is it was related to something not getting installed during a VMware Tools upgrade. I uninstalled VMware Tools, rebooted and installed the VMware Tools. Now VDR works and I can take snapshots with no memory and quiesce.

Reply
0 Kudos
RParker
Immortal
Immortal

If the service " VMWARE Snapshot Provider" don't appear in the list of windows services, we have the same error (-3960).

Right so you need to uninstall the tools, then reinstall the tools and make sure you use custom install so that service appears on the list of components to be installed.

If that service isn't there, it will never work, since that's the service vm ware agents need to communicate to do backups properly.

Reply
0 Kudos
RParker
Immortal
Immortal

Any updates on this at all? I have a similar situation

It's real simple, scroll up a couple of lines, notice the message about:

If the service " VMWARE Snapshot Provider" don't appear in the list of windows services, we have the same error (-3960).

If you don't see it, you need to uninstall / reboot / reinstall vmtools.

and make sure during the install, this service gets installed.

Reply
0 Kudos
habibalby
Hot Shot
Hot Shot

I faced this issue with my Exchange 2007 Passive Node and I have done all the recommendation mentioned in this post and other KB's, but it didn't work. My workaround was to shudown the VM and run vDR back on it and it works fine.

Best Regards,

Hussain Al Sayed

Revisit your posts and award points for "correct" or "helpful".

Best Regards, Hussain Al Sayed Consider awarding points for "correct" or "helpful".
Reply
0 Kudos
sebdelyon
Contributor
Contributor

I've encountered the same problem since I upgraded my VM with the last WM tools package.

All the windows 2008 R2 x64 can't be backup. The service "VMWARE Snapshot Provider" doesn't appear in the service list even if I reinstall vmtools.

Strangly, for windows 2003, It works well.

There is obviously a real problem with Win 2008 R2.

Reply
0 Kudos
mhartmann
Enthusiast
Enthusiast

I had the same issues and solved it by mounting vmware tools, starting setup, choosing "Modify".

Deselect "Filesystem Sync Driver", next... reboot.

No more complaints from Data Recovery. But keep in mind that Data Recovery cannot ensure that the filesystem in your backup will be in a consistent state without these drivers.

But what is more important? No backup or a backup without being sure the the filesystem state is consistent?

I had 3 VMs that never did backups because of that error.

Reply
0 Kudos