Problem with time on VM's after system time changed

[phallinan]

Hi Everyone,

i am just hoping someone can shed a bit of light on a huge problem i had last week. We have 4 x ESX 3.0.1 boxes that have not had any patch's installed (yeah i know bad me). The system time on all of the boxes was way out of wack so we decided to start using NTP on esx. We followed the guide and everything was fine NTP worked well. The VM's are all Windows 2003 and the main dc has it's time manually updated on a daily basis by the admins and then pushes it to the other DC's then the clients which are XP Pro. We do not use the time sync with host in VMtools. About 20 minutes after the change was made the DC's time was updated automaticlly and end up about 7 hours in the future. This was then propergated thru to the other DC's and then the rest of the domain. We had vmotioned machines after the time was updated to try and force any time syncing that may occure but we seen no problems.

What i want to know is how the time was changed on the 1st DC to start with? is there an offset between the virtual cmos and the esx system time? Has this happened to anyone else before?

Thanks

[taylorb]

Pretty much the same scenario happened to me. I popped NTP on my esx hosts and all of a sudden some (not all) of my VMs are 7 hours off. No vmtools time sync checked either. I manually corrected the times on the DCs and everything seemed to fix itself by the next day. No explanation as to why, but I guess it wasn't just a fluke for either of us.

[alhamad]

To avoid future issues, I would suggest to sync the domain PDC using vmware tools. This will gurantee accurate time and will elimnate the need for an admin to do it everyday.

You would also need to configure the PDC to provide time (NTP) but not to sync it with other sources by configuring Type to "NoSync" in HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters

[phallinan]

The reason for setting up NTP was to sync the VM's with the ESX server. Hopefully a VM tech will see this post and have an answer for us both. The reason i am after an answer so much is because it had a huge impact on our prodution site and we lost about 4 hours of production (almost $200K). So managment want a reason and i want to give them an exact answer.

[phallinan]

Taylor i was just wondering what version of ESX you had the problem with? Like i said ours is 3.0.1 unpatched

[taylorb]

3.0.1 unpatched here as well. I have one ESX host that is 3.0.2, but I wasn't running any DCs on it, so I am not sure if it had similar problems. We've been fine since the initial time change. Luckily, We didn't have any major outages other than a few users having a hard time logging in. Management doesn't really even know it happened.

[phallinan]

Just wondering when you had your problem?

[taylorb]

The monday after the last DST change. My ESX hosts were actually several hours off, not just 1, so I gfigured it was time for NTP.