VMware Cloud Community
armaniseal
Contributor
Contributor

Using Snapshots on Domain Controllers safe or not - depends on situation??

A lot has been said about problems with using snapshots with Domain Controllers and most of what has been said simply translates to

"DO NOT DO IT!"

However, based on how things work and why the problems happen I am not sure that this statement applies to all cases.

USN rollback is caused when you revert a single DC back and the USN number of the local copy is lets say 10 but on its replica partner has the reverted DCs USN of 20. Some people ask the question "well how is it that when i restore a DC from a backup i bring it to same state reverting USN and this is not an issue" for one the database invocation ID is changed on restore and the system marks the server and database as restored and thus knows how to proceed with proper replication. This is obviously not the case with snapshot revert.

Now for questions I have to confirm my theory:

CASE 1

so what happens when you have a single DC environment or SBS system? I would this that in the single DC system the snapshots should be perfectly ok for Domain Controller as it has no replication partner....Can anyone at VMware confirm this??

CASE 2

Also what if you have 2 DC's and you want to use snapshot prior to an upgrade and do the following:

1. shutdown replication between servers using "repadmin /options +DISABLE_OUTBOUND_REPL"

2. Take a snapshot of both servers and enable replication back.

and then if something goes wrong you should be able to safely revert backup IF and ONLY IF you revert BOTH DC's back to the same state and enable replication back as it was disabled during snapshot.

Above 2 cases i would think should be ok and would not cause any USN rollback issues or cause any problems on the domain. Can anyone confirm this as i have not had the chance to test it and even if i do you never really know 100% if any damage occurs in database level even if all checks come back good.

Reply
0 Kudos
14 Replies
HendersonD
Hot Shot
Hot Shot

I cannot answer your questions but I do have some input. For awhile we did run snapshots on our domain controllers. These snapshots were done in the middle of the night and about every 3-4 days I needed to restart the domain controllers. The sympton was end users could login fine but my login script would not run properly and drive mappings were missing. As soon as I stopped snapshots all was well.

Reply
0 Kudos
armaniseal
Contributor
Contributor

What OS version was it and also were you Quiesce the VMs when snapshotting? I find that if you have Quiesce enabled it can cause problems for you and it provides absolutely no benefit on AD database. In fact many people I know that had VSS errors in AD when vRanger backup was kicking off was due to this and errors were gone when removing Quiesce option.

Reply
0 Kudos
HendersonD
Hot Shot
Hot Shot

Win2003 R2 Standard. I am using Netapp's SnapManager for Virtual Infrastructure. It can be set to do a VMWare snapshot first, then a Netapp filer snapshot, and then delete the VMWare snapshot. It can also be set to just do a Netapp snapshot only. I ran into trouble when I was doing the VMWare snapshot, it kept doing bad things to my DCs.

I have three DCs, two in my main data closet and one in my DR site. If one has some real bad problem, rather than trying to restore from a snapshot and run into possible issues, it is just as easy to bring up another DC from scratch.

Reply
0 Kudos
mittim12
Immortal
Immortal

I've seen several different issues when a snapshot was created on a domain controller. The one that comes to mind first was users couldn't authenticate until the DC was rebooted. This was several years ago and utilizing Server 2003. We don't do it in our environment so I can't speak to how it behaves with 2008 but I would just avoid it.






If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points

Reply
0 Kudos
armaniseal
Contributor
Contributor

Snapshots themselves should have no bearing on your DC's function. Not unless you revert. I am a Senior Consultant with 46 full time outsourced clients and over 100+ block support clients. I have SAME model i use for backups across board using vRanger and it takes a snapshot "WITHOUT" quiescing the VM. I have NEVER had a problem with any client or their AD due to this. I know AD inside and out which is why i post this because based on the problem the two cases should not have issues. Now mind you that i ALWAYS run a ntbackup of system state scheduled to run prior to vRanger so if i have to restore DC from vRanger i can and then restore system state.

I hear people say its easier to promote a new one. Well it is and its not. What if you have other components on there like the CA or license server. Also if you want to keep the same name of the DC, god i hope no one replies "use ntdsutil to cleanup metadata" as that should not be a task to do on a day to day and should be reserver only for when you can't properly demote DC or restore as no matter how good it is your still ripping into the database and removing objects in an unnatural way which is why there is 10 other tasks you need to do along with that.

I am curious what made you conclude that snapshots were the cause of your issue? What troubleshooting was done? Why could they not login:

was netlogon paused

was the DC advertising

was there issues with secure channel

was there DNS issues

etc..

I am wondering what led to assumption it was Snapshot and not something else... (I am not saying it was not snapshots dont get me wrong, but i work on facts and am curious to find out what happened with your system and what led you to conclude that?)

Reply
0 Kudos
mittim12
Immortal
Immortal

In our situation there was no in depth troubleshooting. The outages we experienced coincided with a snapshot being created on the box every time. We already had an agent based backup system in place for physical servers, exchange, and SQL so we simply moved domain controllers over to that. I honestly haven't given it a second though since then though interesting enough I believe we were running vRanger at the time. I guess it's completely plausible that we had quiescing enabled on that particular job.

.






If you found this or any other post helpful please consider the use of the Helpful/Correct buttons to award points

Reply
0 Kudos
vontexx
Enthusiast
Enthusiast

Hi,

read this carefully!!:

The only solution with Consistent backups of Domain Controller, SQL, Exchange and Co., on Windows 2003 and 2008, you must buy Veeam Backup !!

Otherwise take a "System State" Backup with ntbackup before you backup the DC with a other solution. When you restore a DC, make sure that you boot first into the "Directory Recovery Mode" and restore the "System State"!

Reply
0 Kudos
HendersonD
Hot Shot
Hot Shot

I am sure that it was queuescing the DC that caused the problem. How can I take a VMWare snapshot without queuescing?

Reply
0 Kudos
armaniseal
Contributor
Contributor

TOO FUNNY vontexx! Doing a little advertising on the side are we :)...

But you are right! I know the product and your right. There is a difference between VSS aware "backup" and VSS aware restore. Veeam does this and esentialy maks the database as being restored and not just a file level restore. THis is why it will cause the VM to reboot a couple times. Its been a long time (months) since i looked into this with their product but if i recall thats how they achieve it. Since i use vRanger i just have a ntbackup that does system state before the vRanger kicks off and if i restore it I reboot in DS Restore mode and restore from system state and reboot.

Sorry i know i am posting months after original post but i don't sign in often. not as often as I'd like. Being a Principle Consultant means i barely have time to sleep.

Thanks Guys for all your input and sharing though.

P.S. HendersonD, in your backup you have option to Quiesce the VM or when you take snapshot look on bottom you have the check box.

Reply
0 Kudos
armaniseal
Contributor
Contributor

This was more less discussion not a question. Wanted others input and to share.

Reply
0 Kudos
DSTAVERT
Immortal
Immortal

So here's my thought.

For a smaller organization with a virtual environment I have suggested that they do away with a second domain controller. It is easy to clone the DC (keep it small and simple) and do daily, or more often, system state backups. Smaller organizations often get into more trouble with DC synchronizing, or recovering master roles when they loose one DC.






Forum Upgrade Notice - the VMware Communities forums will be upgraded the weekend of December 12th. The forum will be in read-only mode from Friday, December 10th 6 PM PST until Sunday, December 12th 2 AM PST.

-- David -- VMware Communities Moderator
Reply
0 Kudos
Goatie
Enthusiast
Enthusiast

Hi guys,

You're saying to do snapshots of AD without the Quesce mode, but isn't AD VSS aware (along with SQL, Exchange & sharepoint being about the only other products that are aware)?

My biggest troubles have been with AD when I DON'T quesce the server at snapshot time, rather then doing so!

In ESX 3.5 you could not VSS snapshot and therefore I used Backup Exec System Recovery which did so. Now that I can VSS snapshot I've not had any issues.

Cheers,

Steve

Reply
0 Kudos
anthonymaw
Contributor
Contributor

Reverting a Domain Controller snapshot, in a multi-DC environment, to an earlier point in time is no different than if the server had been powered off for a while and booted up again.

The member DC will contact it's peer DCs and see that its USN is lower and initiate a full replication sync.

Snapshots are best done when VMware Tools triggers Windows Volume Shadow Copy Services to quiesce the Active Directory database write operations.

However the AD ESE database engine is very robust and restoring a "dirty" snapshot without VMware Tools/VSS quiescing generally causes no problems either.

Its just like if the server suffered a power failure, stayed off for a while before rebooted.

Reverting a snapshot should not be confused with restoring Active Directory from a backup, like if you accidentally deleted an object.

The only issue is restoring a snapshot more than sixty days old because some previously deleted AD "tombstoned" objects might reappear.

So it's best not to revert DC snapshots more than 60 days old.

Reply
0 Kudos
scott28tt
VMware Employee
VMware Employee

This thread is 10 years old, I would hope by now the question was answered.


-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog
Reply
0 Kudos