We have two domain controllers running on a 3.0.1 host. One DC is for our root domain and the other is for a child domain. I have set the windows time service to not update via the NoSync option in the registry and have enabled the option for the DC to sync time with the COS. Twice now, we have had both DC's change their time to something pretty random about 5 hours off. The first thing I thought was maybe it was time zone related, however it's not an even number of hours off, it's something random like 5 hours and 12 minutes.
It doesn't seem to be drifting, it just seems to all of the sudden change the time. When it does, we start seeing kerberos issues, which is how we find out things have changed. This didn't seem to be an issue before when I didn't have the NoSync option set and the VM to sync with the host.
Any thoughts of where I could look to fix this issue?
Thanks.
Make sure you have NTP set up on all the ESX hosts.
I should have mentioned that in the original post. NTP is set on all our ESX hosts. The COS time is right.
Do you by chance have any time sync options set in the registry that may be causing this, HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time
http://support.microsoft.com/kb/816042
Have you checked your event logs for any clues?
Also see some of these links...
Configuring windows time service (not in ESX) - http://www.windowsnetworking.com/articles_tutorials/Configuring-Windows-Time-Service.html
Vmware time sync and windows time service (understand windows time first) - http://kb.vmware.com/kb/1318
Time sync - http://www.vmware.com/community/thread.jspa?forumID=21&threadID=16115&messageID=186017
Timekeeping in VMware virtual machines - http://www.vmware.com/pdf/vmware_timekeeping.pdf
Virutalization of Active Directory - http://www.vmware.com/community/thread.jspa?messageID=352424񖂨
Considerations when hosting Active Directory domain controller in virtual hosting environments - http://support.microsoft.com/kb/888794
Virtualizing a Windows Active Directory Domain Infrastructure - http://download3.vmware.com/vmworld/2006/tac9710.pdf
fyi...if you find this post helpful, please award points using the Helpful/COrrect buttons...thanks
The only option set in the registry is the NoSync option. I did look a little deeper in the event log and found that the time got off when I applied patches a few days ago and did a reboot of the server. It looks as though before the reboot, the time was correct, then after the reboot (which in the event log is like 5 hours later where there should have only been a few minute gap) the time was changed to the wrong time. That is when we started seeing the kerberos and other time related issues in the event log.
Why would the server start with a time totally different than that of the COS? Also, shouldn't the VMWare tools correct the wrong time on it's own? Or is the time too far off for the tools to correct?
VMware tools should definitely sync it, Is the DC you are setting this up on the PDC Emulator? Do you have any other DC's that are setup to sync from other sources. You should only have your PDC setup to sync from VMware tools and not the other DC's/servers. The other DC's should sync from the PDC. Just like it mentions here:
To guarantee appropriate common time usage, the Windows Time service uses a hierarchical relationship that controls authority, and the Windows Time service does not permit loops. By default, Windows-based computers use the following hierarchy:
All client desktop computers nominate the authenticating domain controller as their in-bound time partner.
All member servers follow the same process that client desktop computers follow.
All domain controllers in a domain nominate the primary domain controller (PDC) operations master as their in-bound time partner.
All PDC operations masters follow the hierarchy of domains in the selection of their in-bound time partner.
In this hierarchy, the PDC operations master at the root of the forest becomes authoritative for the organization. We highly recommend that you configure the authoritative time server to gather the time from a hardware source. When you configure the authoritative time server to sync with an Internet time source, there is no authentication. We also recommend that you reduce your time correction settings for your servers and stand-alone clients. These recommendations provide more accuracy and security to your domain.
This is not the PDC Emulator. I had originally had it set to get it's time from the domain (the PDC Emulator), but changed it due to VMWare's KB article on time sync and DC's.
Is the best practice to only use the nosync option on the PDC Emulator and let all the rest of the VM's get their time from the domain through the normal hierarchy?
Yes I would do it that way, the PDC Emulator is suppose to authoritative for the whole domain, also check your other DC's to make sure they are not set as authoritative.
The PDC Emulator DC is the only one set as the authoritative time server. I have set my virtualized DC's to no longer sync with the host using VMWare tools. I'm pretty certain this will fix the problem.
What still throws me off is that the event log entries from the reboot I did the other day all started off several hours off. The server did not boot with the correct time and then get changed to the wrong time, it started wrong. Any ideas what would cause the VM to boot with a different time than what is set on the COS? I have made changes so that this will no longer be an issue, however there should be no reason why we can't do it either way.
Is your time zone set correctly on the ESX server that it is syncing from check the /etc/sysconfig/clock file with nano or vi.
Yes. Just double checked and the time zone is correct. The COS time and date is correct and is getting it's time from stratum 2 ntp servers.
If you go to the date/time properties on the DC, do you have it set to automatically synchronize on the Internet Time tab? Is your DC's time zone set correctly?
No internet time tab due to the server being a domain member. The time zone is right.