CIM locks ESXi root account

[imperial69]

Hello there,

need your input for this.
I am running ESXi 6.5 u3 (Fujitsu-VMvisor-Installer-6.5-13932383-v430-1) current patch level 17477841 and get these events:

"Remote access for ESXi local user account 'root' has been locked for 900 seconds after x failed login attempts."

It comes and goes - it totally stops when I stop hosts CIM server. Can't  leave it in stopped state though because its needed for hardware monitoring. 

Did already some research but now I'm stuck...

pam_tally2 --user root
Login Failures Latest failure From
root 1 07/21/21 08:54:04 unknown

I mean - come on - source unknown? Not helpful at all.
Tried to see password related events in hostd.log (altered userx and xxx.xxx.xxx.xxx - these belong to our hardware monitoring system). No declined passwords...

less /var/log/hostd.log |grep -i 'password'
Accepted password for user userx from xxx.xxx.xxx.xxx
Accepted password for user root from 127.0.0.1
Accepted password for user root from 127.0.0.1
Accepted password for user root from 127.0.0.1
Accepted password for user userx from xxx.xxx.xxx.xxx
Accepted password for user root from 127.0.0.1
Accepted password for user root from 127.0.0.1
Accepted password for user root from 127.0.0.1
Accepted password for user userx from xxx.xxx.xxx.xxx
Accepted password for user root from 127.0.0.1
Accepted password for user userx from xxx.xxx.xxx.xxx
Accepted password for user userx from xxx.xxx.xxx.xxx
Accepted password for user userx from xxx.xxx.xxx.xxx

No other IP addresses at the time in rhttpproxy.log either. 
Luckily, no attack but something "internal". But what is it?

What would be your next steps? Would be glad, if you point me into the right direction...
Thanks!

[imperial69]

Seems to be a difficult one 😉

Just wanted to let you know - I contacted the German FUJITSU community as well.

Any hint would still be welcome of course!

[Srijithk]

can you try the workaround in this KB --> https://kb.vmware.com/s/article/67920

Also, you need to look at auth.log to identify the culprit

Thanks,

Srijith, 

[nachogonzalez]

Hello, hope you are doing fine.

Quick question: have you reset the password lately?


can you please give a test to the KB mentioned avobe

[imperial69]

Sorry for my delayed response, I had some days off 🙂

I really appreciate your input.
First of all: No, we did not change servers root password.

Mentioned solution (https://kb.vmware.com/s/article/67920) is interesting but I don't think it applies to my issue. Here are my reasons:

• KB solution states, this has been fixed with ESXi 6.5 U3... We already use this version since late 2019.
• Appart from the "...account 'root' has been locked" in vobd.log we don't see the symptoms mentioned in this KB.
  • no "Out of HTTP sessions" in hostd.log
  • no "admission failure" entries in vmkernel.log during "account locked" time window
  • nothing for account root in vmsyslogd-dropped.log

Do you think I should try the workaround anyway and edit servers hostd/config.xml?

Log auth.log is not really helpfull. We had these account issues again on 27th and all it says for the day is:

2021-08-27T08:34:21Z sshd[418292556]: /etc/ssh/sshd_config line 6: Deprecated option UsePrivilegeSeparation
2021-08-27T08:34:21Z sshd[418292556]: /etc/ssh/sshd_config line 21: Unsupported option PrintLastLog
2021-08-27T08:34:27Z sshd[418292556]: Accepted publickey for root from xxx.xxx.xxx.xxx port XXXX ssh2: XXX...
2021-08-27T08:34:27Z sshd[418292556]: pam_unix(sshd:session): session opened for user root by (uid=0)
2021-08-27T09:16:22Z sshd[418292556]: pam_unix(sshd:session): session closed for user root

That was me checking the logs... (I altered IP and other security related stuff in the snippet). Yes, I could clean up sshd config...

I have to admit that I took over this machine (and others) recently. Now this annoying issue is mine to deal with.

Once again: Thanks @Srijithk and @nachogonzalez!
Greetings!