Account Lockout via vCenter

[BonezOz]

I have a user in our domain that is constantly being locked out of our AD.  Now normally this would be put into the TechNet forums except for one thing, the bad password is being passed through vCenter.  I've been able to trace a lot of information through our AD name servers, all the way to the vCenter logs:

[2012-08-27 15:15:37.574 01764 info 'Libs'] [ADS] Account username found, but not local
[2012-08-27 15:15:37.574 01764 info 'App'] Error 1326 authenticating user .\username.
[2012-08-27 15:15:37.589 01764 info 'App'] Error 1326 authenticating user DOMAIN\username.
[2012-08-27 15:15:37.589 01764 error 'App'] Failed to authenticate user <DOMAIN\username>
[2012-08-27 15:15:37.621 01764 info 'App'] [VpxLRO] -- FINISH task-internal-1333746 --  -- vim.SessionManager.login -- 739C576E-A884-41F9-A4ED-BAFF0F8E0AAC
[2012-08-27 15:15:37.621 01764 info 'App'] [VpxLRO] -- ERROR task-internal-1333746 --  -- vim.SessionManager.login: vim.fault.InvalidLogin:
(vim.fault.InvalidLogin) {
   dynamicType = <unset>,
   faultCause = (vmodl.MethodFault) null,
   msg = "",
}

This is occuring every 2:30hrs and 7:30hrs, which tells me that it's either a scheduled task that is occuring somewhere, or a system monitor that polls at these times, however I can't find either anywhere.

What I'm trying to figure out is if there is a way to trace the failed login attempt to a specific IP address?  The Security Logs for the server are set to forward to the name server and that only points to the vCenter server.

Other points:

Windows Server 2003 R2

vCenter Server 4.0.0 258672

ESXi 4.0.0 261974 (2 servers clustered)

VMware View 4.5.0 build-293049

[BonezOz]

I've discovered a bit more information in regards to my issue.  The user recently reset his domain password to one that he used previously.  Now he's no longer getting locked out and vCenter is registering a successfull login.  Trouble now is I need to find out which application or why he's got a login set up on this server and where it's coming from.  All logs point that the login is coming from within the vCenter Server, but there doesn't appear to be an application or a reason as to why this is occuring, there isn't even a scheduled task within vSphere that is being called.

This issue is really baffling me, I thought for sure with the successful login's that I'd be able to discover the reason.  I've been running process monitor, Wireshark, and process explorer in order to track the logins and applications, but the only thing I've found is the logins, and something called ariliameter on port 62306.  There are multiple entries in Wiresharks logs for this until the users account logs out about 3 or 4 seconds later.

Anyone with any ideas?

[mrksiddiqui]

Are there any thrid party plugins that were installed by that user account and any applications like Veem that help with VMware? At the same time I would install ALockout.dll on the vecenter to confrim vCenter or its process is casuing lockout.

http://www.microsoft.com/downloads/details.aspx?FamilyId=7AF2E69C-91F3-4E63-8629-B999ADDE0B9E&displa...

[BonezOz]

Thank you, I think I may have found the answer.  There were 3 Xantagi virtual appliances running in the environment.  I've now shut them down as I discovered that they were no longer necessary or used.  I have to wait for a little while to see if this is where the users lockout/login issues were coming from.

[BonezOz]

Finally found the issue, after months of searching and investigating.

Some history first:

One user within our IT team kept getting locked out of his account on a regular basis.  Initial investigation showed that this was happening from and on one of our vCenter Servers.  But after looking at scheduled tasks and not seeing one that used his credentials deeper investigation had to occur.  First was login debugging within windows, again this proved fruitless. Next I checked the vpxd logs within vCenter and though this showed the logins attempting to occur it failed to show what was calling them.

After a time I resorted to Sysinternals Process Monitor, process explorer and Wireshark.  These did prove a bit more indepth and I could see the failed login attempts, but still no source.

A break came when the user reset his password back to an older password.  The lockouts stopped occuring!

Running process monitor and wireshark again I started tracking down where the login were occuring from.  It appears that over a year ago the company was looking to deploy VMware View across the south pacific (Australia, New Zealand, southern China) and had installed 3 Xangati servers to monitor how the performance would be across several countries from a central point.  After discussing these servers I found that no one was using them and no one knew what the usernames or passwords to login were.  I was told then to go ahead and shut them down.

After an hour of shutting them down the login of the user was to occur again, because it had appeared through investigation to happen at specific times.  At the specified time there were no login attempts on the vCenter server, no records in the Netlogon.txt file and no entries in Wireshark or Process Monitor.

The case was solved!

There is a moral to this story, one that anyone working with any system should take to heart.  Never, ever forget what systems you've used your personal login details to create scheduled tasks.  Always use a service account with a password set to never expire.  Besides, what would happen if you left the company and your account was used to run critical system tasks?  A deleted account would cause these to fail.

TTFN and thanks for those that had a look at this topic.