Hello everyone
I got the following message every hour:
"Remote access for ESXi local user account 'root' has been locked for 120 seconds..."
I found a lot of information how to figure this out:
Security.AccountLockFailures. Maximum number of failed login attempts before a user’s account is locked. Zero disables account locking.
Security.AccountUnlockTime. Number of seconds that a user is locked out.
But I didn't get out where to find the source IP who is trying to access our ESXi hosts. Isn't it in /var/log/auth.log ?
Thank you for your input.
HI ,
Are you using any management tools on your environment .
This issue typically comes if any third party monitoring software that is unable to login and locks out the root user.
Use any monitoring tools slike Orion SolarWinds or TCPDump to identify product/source which is causing the issue.
It's a customer environment, so I don't know exactly what they have for monitoring tools. That is why I'm asking for the file who contains the source IP. and such failed login logs.
The log file you are looking for is /var/log/auth.log.
More information can be found in this KB article, https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=20048...
Hi ,
Try below
Login to the ESXi console and ran this command
# less /var/log/auth.log
You can see below example when I given wrong password second time it triggered error message
2017-03-02T08:30:57Z sshd[751607]: Connection from 192.168.1.50 port 55000
2017-03-02T08:31:02Z sshd[751607]: Accepted keyboard-interactive/pam for root from 10.20.2.17 port 55000 ssh2
2017-03-02T08:31:02Z sshd[751607]: pam_unix(sshd:session): session opened for user root by (uid=0)
2017-03-02T08:31:02Z sshd[751610]: Session opened for 'root' on /dev/char/pty/t0
2017-03-02T08:31:34Z sshd[751644]: Connection from 192.168.1.50 port 55075
2017-03-02T08:31:39Z sshd[751645]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.20.2.17 user=root
2017-03-02T08:31:41Z sshd[751644]: error: PAM: Authentication failure for root from 192.168.1.50
I happened to have to same issues on all my ESXi hosts roughly one month after Upgrading from ESXi 5.5 to ESXi 6.5.
I upgraded using Updating Manager and went from a Lenovo ESXi image for my IBM servers to a VMware generic ESXi image.
/var/log/syslog.log revealed a vendor script repeatedly running and failing.
Revoking the script from scheduled tasks (cron) fixed the issue.
You may want to have a look at your cron jobs with vi:
$ vi vi /var/spool/cron/crontabs/root
You can check the event which gives you the IP address and also there is a new feature in 6.0 which has default account lockout. Pls find below URL for more info https://www.v-front.de/2015/04/watch-out-esxi-60-introduces-root.html?m=1
Thanks,
MS
I am too having this this issue but just one of many hosts. I am not sure if it is a DELL plugin or a SOLARWINDS plugin and since it does not tell me the IP I cant' find what is causing it. I checked the ESXi shell and it is not enabled.
check the vobd.log in the ESXi host after taking putty session and it should give the ip address or check the hostd.log which gives the login failure with ip address
Thanks,
MS
I keep seeing this...
I keep seeing this and getting lockouts
2017-10-16T07:51:40Z ^T: pam_unix(openwsman:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost= user=root
2017-10-16T07:51:43Z ^T: pam_tally2(openwsman:auth): user root (0) tally 7869, deny 5
2017-10-16T07:51:43Z addVob[1256326]: Could not expand environment variable HOME.
2017-10-16T07:51:43Z addVob[1256326]: Could not expand environment variable HOME.
2017-10-16T07:51:43Z addVob[1256326]: DictionaryLoad: Cannot open file "/usr/lib/vmware/config": No such file or directory.
2017-10-16T07:51:43Z addVob[1256326]: DictionaryLoad: Cannot open file "~/.vmware/config": No such file or directory.
2017-10-16T07:51:43Z addVob[1256326]: DictionaryLoad: Cannot open file "~/.vmware/preferences": No such file or directory.
I have the same problem, and can't find any log that contains the SOURCE of that failed logins! :smileyshocked:
vobd.log only contains the message "Remote access for ESXi local user account 'root' has been locked for 120 seconds" over and over, but no log file seems to have the source IP... how is it possible to find the problem?!
Try rhttpproxy or hostd log for information same time as the vodb states"Remote access for ESXi local user account 'root' has been locked for 120 seconds... rhttpproxy should give you IP address
Thanks,
MS
Hello
I have the same problem.Do you find a solution to the problem ? Please tell me the solution,thank you.
Hi all,
I have the same issue and I can't figure out what is going...
It happened to me to a fresh installation of ESXi 6.5 U2 - no monitoring tools etc. to the server, just configured an external syslog (Log Insight) to collect logs.
Any further ideas?
Thx
T.
I had the same issue.
I was able to find the IP address via vCenter > click host > Monitor > Tasks & Events > Events. Each failed logon would show the warning you see on the main summary screen, but also the failed logon attempt like "root@<source IP address>. In my case, I wasn't familiar with the IP I was seeing (likely legacy stuff, this popped up after a root password change), so I modified the firewall settings for SSH and vSphere Web Access to only trust the IP ranges I actually trust. 900 seconds later, good to go, warnings cleared.
IP redacted below, but you get the idea. I believe this was added to vSphere 6.x, FWIW.
Can I get a solution for this:
ESXi root password is getting locked frequently
ESXi root password is getting locked frequently
I found that no machine/agent is used to authenticate ESXi server:
I rebooted ESXi several times.
Same issue.
Earlier the ESXi version was:
VMware ESXi, 6.5.0, 8294253
I even upgraded ESXi to patch but still seeing same issues:
VMware ESXi 6.5.0 build-16576891, Update 3
[root@btp01esx16:/var/log] pam_tally2 --user root
Login Failures Latest failure From
root 250 09/14/20 05:29:57 unknown
[root@btp01esx16:/var/log] pam_tally2 --user root --reset
Login Failures Latest failure From
root 250 09/14/20 05:29:57 unknown
I see following lines are continuosly recorded on /var/log/auth.log
2020-09-13T04:49:32Z sshd[71851]: rekeyed inbound cipher
2020-09-13T05:49:32Z sshd[71851]: rekeyed outbound cipher
2020-09-13T05:49:32Z sshd[71851]: rekeyed inbound cipher
2020-09-13T06:49:33Z sshd[71851]: rekeyed outbound cipher
2020-09-13T06:49:33Z sshd[71851]: rekeyed inbound cipher
2020-09-13T07:49:33Z sshd[71851]: rekeyed outbound cipher
2020-09-13T07:49:34Z sshd[71851]: rekeyed inbound cipher
2020-09-13T08:49:34Z sshd[71851]: rekeyed outbound cipher
2020-09-13T08:49:34Z sshd[71851]: rekeyed inbound cipher
2020-09-13T09:49:35Z sshd[71851]: rekeyed outbound cipher
2020-09-13T09:49:35Z sshd[71851]: rekeyed inbound cipher
2020-09-13T10:49:35Z sshd[71851]: rekeyed outbound cipher
2020-09-13T10:49:36Z sshd[71851]: rekeyed inbound cipher
2020-09-13T11:49:36Z sshd[71851]: rekeyed outbound cipher
2020-09-13T11:49:36Z sshd[71851]: rekeyed inbound cipher
2020-09-13T12:49:37Z sshd[71851]: rekeyed outbound cipher
2020-09-13T12:49:37Z sshd[71851]: rekeyed inbound cipher
2020-09-13T13:49:37Z sshd[71851]: rekeyed outbound cipher
2020-09-13T13:49:38Z sshd[71851]: rekeyed inbound cipher
2020-09-13T14:49:38Z sshd[71851]: rekeyed outbound cipher
2020-09-13T14:49:38Z sshd[71851]: rekeyed inbound cipher
2020-09-13T15:49:39Z sshd[71851]: rekeyed outbound cipher
2020-09-13T15:49:39Z sshd[71851]: rekeyed inbound cipher
2020-09-13T16:49:40Z sshd[71851]: rekeyed outbound cipher
2020-09-13T16:49:40Z sshd[71851]: rekeyed inbound cipher
2020-09-13T17:49:40Z sshd[71851]: rekeyed outbound cipher
2020-09-13T17:49:41Z sshd[71851]: rekeyed inbound cipher
2020-09-13T18:49:41Z sshd[71851]: rekeyed outbound cipher
2020-09-13T18:49:41Z sshd[71851]: rekeyed inbound cipher
2020-09-13T19:49:42Z sshd[71851]: rekeyed outbound cipher
2020-09-13T19:49:42Z sshd[71851]: rekeyed inbound cipher
2020-09-13T20:49:42Z sshd[71851]: rekeyed outbound cipher
2020-09-13T20:49:42Z sshd[71851]: rekeyed inbound cipher
2020-09-13T21:49:43Z sshd[71851]: rekeyed outbound cipher
2020-09-13T21:49:43Z sshd[71851]: rekeyed inbound cipher
2020-09-13T22:49:44Z sshd[71851]: rekeyed outbound cipher
2020-09-13T22:49:44Z sshd[71851]: rekeyed inbound cipher
2020-09-13T23:49:44Z sshd[71851]: rekeyed outbound cipher
2020-09-13T23:49:45Z sshd[71851]: rekeyed inbound cipher
2020-09-14T00:49:45Z sshd[71851]: rekeyed outbound cipher
2020-09-14T00:49:45Z sshd[71851]: rekeyed inbound cipher
2020-09-14T01:49:46Z sshd[71851]: rekeyed outbound cipher
2020-09-14T01:49:46Z sshd[71851]: rekeyed inbound cipher
2020-09-14T02:49:46Z sshd[71851]: rekeyed outbound cipher
2020-09-14T02:49:47Z sshd[71851]: rekeyed inbound cipher
2020-09-14T03:49:47Z sshd[71851]: rekeyed outbound cipher
2020-09-14T03:49:47Z sshd[71851]: rekeyed inbound cipher
2020-09-14T04:49:48Z sshd[71851]: rekeyed outbound cipher
2020-09-14T04:49:48Z sshd[71851]: rekeyed inbound cipher
It could be possible a remote machine is trying to access the host. If you are using a monitoring tool such as vCenter, enable a lockdown of the host so that you can only access the host via vCenter portal or the local host. Below is the steps to do so in vCenter 6.5:
1. Select the host
2. Click on Configure
3. Under System, select Security Profile
4. Under Lockdown Mode, select Edit
5. Choose Normal and ok