NTP issues on ESXi

[unsichtbare]

I have a lab environment with the following:

• ESXi 6 U2 - joined to the domain
• NTP Client running and functioning (see ntpq -p results below)
  • 0.us.pool.ntp.org, 1.us.pool.ntp.org, 2.us.pool.ntp.org
• Windows DC with VMware tools time (default settings)

Last night there was a short power outage around 21:15 8/15/2016, and this lab is not protected by backup power. This morning, all hosts and the DC were reporting that the time was around 23:30 8/15/2016. Network and internet connectivity were available.

My question: How did the ESXi hosts get so far off when they are set to pool.ntp.org?

My second question: Does joining an ESXi Host to the domain supersede the NTP configuration for time? Because after I set the ESXi hosts to the correct time, then set the DC's, the hosts were hours off again!

THX

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
aprihop.cdknnjl 200.98.196.212   2 u   51   64   17   69.636  -98652. 56958.3
ranger.innolan. 64.113.44.57     3 u   51   64   17  125.549  -98627. 56957.6
ec2-52-0-56-137 198.82.247.71    3 u   47   64   17   65.675  -98652. 56957.2

P.S> I believe that the jitter is so high because I just got the time set correctly on ESXi and DC, but first thing I wanted to check was that the when interval was less than the poll, meaning that NTP is at least querying time, if not applying it!

[ubhatti1985]

Hi,

Generally if you are in a Domain environment you point your ESXi hosts to PDC Emulator and PDC Emulator goes out to Microsoft.

Regards
Umar

[pedrocalixto]

If you're in a Microsoft environment, you should check if the PDC Emulator of your Active Directory is configured as a NTP server and not as NT5DS. Also, you should check if the SSH service is starting with the host.

[unsichtbare]

Thanks for the reply, but I have ESXi set to independently obtain time from us.pool.ntp.org  and confirmed that it is doing so using the command: ntpq -p

Part of the rationale here is that the DC's are VMs also, and set to use an external time: https://support.microsoft.com/en-us/kb/816042

and also, VMware Tools Time synchronization has been disabled for the VM: Disabling Time Synchronization (1189) | VMware KB

I guess the question remains, it there an undocumented (I can' find anything) property of joining an ESXi host to an AD Domain that supersceeds the NTP configuration (ntp.conf)?

[unsichtbare]

Thanks for the reply.

PDC is set to NTP, but my hosts are set to use us.pool.ntp.org

SSH is enabled and port 123 is not blocked on the firewall, as I can confirm by running: ntpq -p

Thanks,

[ThompsG]

Hi unsichtbare.

Sorry to be asking a dumb question but what do you mean by "hours off again" - I know really dumb question but just wondering if it could be as simple as the timezone offset?

Cheers.

[unsichtbare]

So after the power outage, the ESXi and all VMs were reading 11:00 the previous night - about 7 hours slow.

Then I corrected the DC's (which corrected all the windows boxes, remember I have the DC's set not to sync using VMware tools.

Then I corrected the ESXi Hosts (which shouldn't have been off at all because NTP was set and working, according to ntpq -p)

Then I noticed the DC's and all VMs were 7 hours ahead! (remember I have the DC's set not to sync using VMware tools)

So I corrected the DC's again, and holy crap, the ESXi hosts were now 7 hours slow!

Eventually, I kept going in a circle, correcting time until everything was correct.

[jlanders · Accepted solution]

>> I guess the question remains, it there an undocumented (I can' find anything) property of joining an ESXi host to an AD Domain that supersedes the NTP configuration (ntp.conf)?

Your intuition is right that both Likewise AD client and ntpd will try to synchronize the system clock. By default, if you don't configure or disable NTP, you will synchronize ESXi's system clock using the DC's time via AD. If you want to do the opposite and use NTP instead of AD to set the system clock, you may find the following KB 1035833 helpful:

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=10358...

Likewise sets the system clock using the time supplied by your DC. And, 'ntpd' sets your system clock using your upstream NTP servers. Generally, when everything is synchronized to the correct time, you might notice only very small time differences on ESXi. These small differences can happen because Likewise sets the system clock to the DC's time, while 'ntpd' will step or slew the system clock to a time derived from your upstream NTP servers.

In this case, it looks like the time supplied to Likewise by the DC wasn't correct. Perhaps the DC had problems synchronizing with your upstream time source. Or, perhaps the maximum phase restriction described in Microsoft's KB article prevented the DC's clock from changing to the correct time. But, both Likewise and 'ntpd' were trying to change the system clock using significantly different values of the current time. As you guessed, you will want to pick one or the other.

[unsichtbare]

Thanks,

I googled and googled and couldn't come up with the KB you cited!!

The root problem was with the Windows DC's and it kept affecting the ESXi hosts, even though I have NTP configured, all because of Likewise!