VMware Cloud Community
AntonVZhbankov
Immortal
Immortal

Windows 2008R2 snapshot problem on 4.1

I have problem with snapshots on one of my VMs. And what's strange it started only last friday.

One of my DC's on Win2008R2 refused to take quiesced snapshot after clod migration from one cluster to another. Remaining DC, absolutely the same configuration still can take snapshots.

I've made new VM, built from nothing, clean installation, promoted to DC instead of failed on and problem still persists.

What I've found in Windows logs:

1) Volume Shadow Copy Service error: Unexpected error DeviceIoControl(
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} - 000000000000042C,0x00560000,0000000000000000,0,00000000003C7F70,4096,[0]). hr = 0x80070001, Incorrect function.

.

Operation:

Exposing Recovered Volumes

Locating shadow-copy LUNs

PostSnapshot Event

Executing Asynchronous Operation

Context:

Device:
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Examining Detected Volume: Existing -
?\fdc#generic_floppy_drive#6&2bc13940&0&0#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}

Execution Context: Provider

Provider Name: VMware Snapshot Provider

Provider Version: 1.0.0

Provider ID: {564d7761-7265-2056-5353-2050726f7669}

Current State: DoSnapshotSet

2) lsass (460) An attempt to write to the file "
?\Volume{b6c3b2c0-a388-11df-badc-0050568b0019}\Windows\NTDS\edb.log" at offset 7525888 (0x000000000072d600) for 512 (0x00000200) bytes failed after 0 seconds with system error 19 (0x00000013): "The media is write protected. ". The write operation will fail with error -1032 (0xfffffbf8). If this error persists then the file may be damaged and may need to be restored from a previous backup.

3) lsass (460) Unable to write to logfile
?\Volume{b6c3b2c0-a388-11df-badc-0050568b0019}\Windows\NTDS\edb.log. Error -1032 (0xfffffbf8).


---

MCSA, MCTS Hyper-V, VCP 3/4, VMware vExpert

http://blog.vadmin.ru

EMCCAe, HPE ASE, MCITP: SA+VA, VCP 3/4/5, VMware vExpert XO (14 stars)
VMUG Russia Leader
http://t.me/beerpanda
Reply
0 Kudos
99 Replies
mrjlturner
Contributor
Contributor

Jcwuerfl,

No floppy disks are connected (at least in my case.) It is entirely related to ADAM or other AD components being installed, as I see the same issues on my DC's that are running 2008 R2 as well.

I'm just hoping that the folks in this thread that have tickets open w/ VMware Support actually make some progress on this one...

-mrjlturner

Reply
0 Kudos
pciccone
Contributor
Contributor

Ahh, gotcha. I just wanted to make sure you knew - just in case of a restore and you might have missed data and pulled some hair out Smiley Happy

Reply
0 Kudos
jcwuerfl
Hot Shot
Hot Shot

Ok, what about a floppy drive though? having a floppy drive present and not having something "connected" to it as its trying to write information to it and it can't ? If you have a floppy drive listed in your VM Edit window, try removing it.

Reply
0 Kudos
mrjlturner
Contributor
Contributor

Pciccone,

I appreciate the looking out for a brother! It's nice to know that at least we as a VMware community have each others' backs... now if we can just get VMware on board with that mentality we should be moving in the right direction!

I found this link while researching this today: http://communities.vmware.com/thread/283944?tstart=60

LucD make a script that will add the disk.EnableUUID key to each virtual machine, which will help us all out when this is finally fixed...

-mrjlturner

Reply
0 Kudos
mrjlturner
Contributor
Contributor

Jcwuerfl,

I apologize for not being more clear. There are no floppy drives present (disconnected or otherwise) on the VMs that are having issues. I've never had a need w/ the newer OS's to have floppy drives attached, so I build them w/o them out of the gate. It is an odd error that it spits out a FDC controller during the VSS process...but the other guy was right...it is spitting out those errors, even though no floppy drives are present.

-mrjlturner

Reply
0 Kudos
BulletByte
Contributor
Contributor

Your post suggests that this problem is fixed by enabling the UUID but thats NOT the truth!

All my vCenter/ESX 4.1 fresh installed datacenter has on really all machines the UUID enabled but anyway, this is not the solution for all out in the field. There must be something more!

Reply
0 Kudos
mronsman
Contributor
Contributor

Indeed that did work changing the UUID to false. Thanks for the help! I understand it does not quiesce the Host OS but something is better than nothing!

I would give you points if this were my original post! Smiley Happy

Thanx,

Matt

Reply
0 Kudos
OmniTech
Contributor
Contributor

This solution is only "good" for VMs with no transactionnal systems. But you could have some corrupted files, beware.

I do hope VMWare will solve this soon enought cause otherwise they should change the Compatibility Matrix for the guest OS...

Reply
0 Kudos
jmerchan
Contributor
Contributor

Same bug!! no solution

Reply
0 Kudos
kswail
Contributor
Contributor

@Omnitech: You are totally correct, but it seems this problem is only happening on transactional systems thus far (AD and ADLDS to be precise).

@All: Since VMware is not forthcoming on this one and no real solution is in sight (reminder, setting the UUID to false is not a solution, and you don't want to find this out during a DR scenario) I've come up with a simple temp solution.

What I've done for now is to create a new VMDK file for my DC and use the Windows Server Backup to backup to that VMDK. At least now I know that I have a consistent backup as long as I backup the VM outside of the Windows Server Backup backup window.

I hope to see some news on a permanent fix soon.

cheers,

kswail

Reply
0 Kudos
MKguy
Virtuoso
Virtuoso

2008R2 Snapshot issues with AD/ADAM databases?

Sounds like this issue to me:

http://communities.vmware.com/message/1596547

We were able to solve with the VMware support for our virtual vCenter Server with it's ADAM instance as described in this thread.

-- http://alpacapowered.wordpress.com
Reply
0 Kudos
lucmorin
Contributor
Contributor

@all,

Just to keep you posted about the status of my SR with VMWare on this subject.

It seems that this porblem is caused by a bad VMWare tools install, as some files that should be installed aren't. One such example is the VCBRequestor.dll file.

At one point, a VMWare tech rep connected remotely and we managed to copy/register this dll (it's more than simply regsvr32ing it, as he also changed some COM+ settings which I don't remember). After properly registering this dll on the guest OS, quiesced snapshots were successfully taken.

I'm still waiting for a "patch", or at least a detailed step by step instructions list on how to register the missing dll, but it seems that those VMWare guys aren't too quick in returning calls/emails. That's very disapointing, and my customer is really starting to question our decision to go with VMWare.

Regards

Reply
0 Kudos
OmniTech
Contributor
Contributor

@lucmorin

Where can you get that specific file?

Where do you copy this file in Windows 2008?

I'll make some test myself.

Thanks

Reply
0 Kudos
lucmorin
Contributor
Contributor

@OmniTech,

Like I mentioned, it's not only a matter of copying the file to the guest. It involves registering the dll, but also changing some settings in the COM+ Windows subsystem. I don't have an exact proceudre for you to follow at this time, but I'm on the heels of the VMWare tech reps to provide me one.

As soon as I have it, I'll post it here.

Do I understand that on your Windows 2008 R2 guest you are also missing the VCBRequestor.dll file, and experiencing the same snapshots problems ? Please confirm, as this will give me more weight when discussing with vmware.

Thanks

Reply
0 Kudos
OmniTech
Contributor
Contributor

@lucmorin

The wierd thing is, I have searched my Windows 2003 VM and I have not found that file... But Snapshots works fine.

Reply
0 Kudos
lucmorin
Contributor
Contributor

@OmniTech,

Is your Win 2003 VM a vCenter server ?

The VCBRequestor.dll file, if I understood correctly what the VMWare people told me, is only used when trying to take a quiesced snapshot on a VM that has an ADLDS role, such as a vCenter server VM.

Also, the fact that snapshots seem to complete normaly doesn't mean that they will be application consistent. Under some circumstances the snapshot will simply "ride through" the abnormal situation, and take a crash consistent snapshot. If I'm not mistaken, one such example is if you install VMWare tools without the VSS driver, and then try to take a quiesced snapshot. It will seem to have succeeded, but the filesystem will not have been quiesced during the snapshot.

In any case, I've recevied instructions on how to register the missing dll, I just need to test things through, and if it works, I'll post the instructions here. I'll also try to setup a download link for the missing dll.

Cheers

Reply
0 Kudos
OmniTech
Contributor
Contributor

@lucmorin

It is a Win 2003 VM but not a DC

So I guess it's normal this file is not present.

But in a Win2008R2 VM, the snapshots still does not work and they are not DCs either.

Reply
0 Kudos
lucmorin
Contributor
Contributor

But is the VM a vCenter servers ?

Reply
0 Kudos
OmniTech
Contributor
Contributor

Yes, build upon a template

Reply
0 Kudos
lucmorin
Contributor
Contributor

When you install vCenter on a VM, it adds the "Active Directory Lightweight Directory Service" role to the guest.

So, when taking a quiesced snapshot, you need to tell this service to complete its transaction and then wait for the snapshot to complete (I'm simplifying to extremes here). In order to achieve that, the VSS infrastructure is used, and the ADLDS needs a "writer", and in our case it's the ADAM writer installed by VMWare tools.

Unfortunately, on WIndows 2008 R2, it seems that some components are not properly installed and registered, thus causing the problem. I have received a procedure to "patch" the problem, and it seems to work, but I need to make syure my snapshot is valid and I just don't have the time to do a full DR right now.

Once I'm sure it works, I'll post the steps and files on my blog (not too sure if I can attach files here).

regards

Reply
0 Kudos