VMware Cloud Community
Cyanatide
Enthusiast
Enthusiast
‎03-31-2020 03:16 AM

VMWare vCenter Single Sign on randomly refuse connection from user due to incorrect login/password

Hi!

I'm using VMware vCenter Server Appliance 6.5.0.5100.

Since few days, almost all my login are refused due to incorrect login/password, but I'm sure of my credentials!

Sometimes login is accepted and if I close session, then I'm sure I will not be able to connect again during 10-20 minutes...

When my login fail, nothing is logged vCenter Server, or in /var/log/vmware/sso/ssoAdminServer.log

First I thought the problem was due to my log partition which was 100% used. I cleaned log, now storage is fine but the problem remains...

Any idea to investigate this problem?

Best regards.

Tags (2)
0 Kudos
6 Replies
berndweyand
Expert
Expert
‎03-31-2020 03:34 AM

logins are also logged in:

/var/log/audit/sso-events/audit_events.log

/var/log/vmware/sso/websso.log

is your vcenter integrated in an ad-domain? is the domain the primary authentication source ?

0 Kudos
Cyanatide
Enthusiast
Enthusiast
‎03-31-2020 04:08 AM

Thanks,

I have nothing in /var/log/audit/sso-events/audit_events.log, but indeed in /var/log/vmware/sso/websso.log, I can see what seems to be an error with tomcat 6.

[2020-03-31T10:58:30.749Z  tomcat-http--46                                       INFO  com.vmware.identity.SsoController] Welcome to ssoSSLDummy handler! The client locale is fr, DEFAULT tenant [2020-03-31T10:58:30.801Z  tomcat-http--46                                       INFO  com.vmware.identity.SsoController] Welcome to ssoSSLDummy handler! The client locale is frvsphere2.local [2020-03-31T10:58:30.815Z  tomcat-http--46                                       INFO  com.vmware.identity.SsoController] Send ssl probing reponse for: https://vcenter2.eona.local/vsphere-client/saml/websso/metadata [2020-03-31T10:58:31.336Z  tomcat-http--16                                       INFO  com.vmware.identity.MetadataController] Welcome to Metadata handler! The client locale is en_US, tenant is vsphere2.local [2020-03-31T10:58:31.876Z  tomcat-http--7                                       INFO  com.vmware.identity.SsoController] Welcome to SP-initiated AuthnRequest handler! The client locale is fr, tenant is vsphere2.local [2020-03-31T10:58:31.876Z  tomcat-http--7                                       INFO  com.vmware.identity.SsoController] Request URL is https://vcenter2.eona.local/websso/SAML2/SSO/vsphere2.local [2020-03-31T10:58:32.285Z  tomcat-http--7  9ad99efa-9d10-4537-a306-b3eb24075ae2 INFO  com.vmware.identity.samlservice.impl.AuthnRequestStateValidator] Authn request proxyCount= null set isProxying=false [2020-03-31T10:58:32.294Z  tomcat-http--7  9ad99efa-9d10-4537-a306-b3eb24075ae2 INFO  com.vmware.identity.samlservice.impl.AuthnRequestStateValidator] Authentication request validation succeeded [2020-03-31T10:58:32.297Z  tomcat-http--7  9ad99efa-9d10-4537-a306-b3eb24075ae2 INFO  com.vmware.identity.SsoController] Server SPN is HTTP/vcenter2.eona.local [2020-03-31T10:58:32.301Z  tomcat-http--7  9ad99efa-9d10-4537-a306-b3eb24075ae2 INFO  com.vmware.identity.SsoController] Accessing Tenant vsphere2.local, brand name string null [2020-03-31T10:58:40.313Z  tomcat-http--6                                       INFO  com.vmware.identity.SsoController] Welcome to SP-initiated AuthnRequest handler! The client locale is fr, tenant is vsphere2.local [2020-03-31T10:58:40.313Z  tomcat-http--6                                       INFO  com.vmware.identity.SsoController] Request URL is https://vcenter2.eona.local/websso/SAML2/SSO/vsphere2.local [2020-03-31T10:58:40.370Z  tomcat-http--6  eb3feb1d-8ce1-4273-b0cf-8bc7076e9396 INFO  com.vmware.identity.samlservice.impl.AuthnRequestStateValidator] Authn request proxyCount= null set isProxying=false [2020-03-31T10:58:40.379Z  tomcat-http--6  eb3feb1d-8ce1-4273-b0cf-8bc7076e9396 INFO  com.vmware.identity.samlservice.impl.AuthnRequestStateValidator] Authentication request validation succeeded [2020-03-31T10:58:40.838Z  tomcat-http--6  eb3feb1d-8ce1-4273-b0cf-8bc7076e9396 ERROR com.vmware.identity.samlservice.impl.CasIdmAccessor] Caught exception. com.vmware.identity.idm.IDMLoginException: Native platform error [code: 851968][null][null]         at com.vmware.identity.idm.server.ServerUtils.getRemoteException(ServerUtils.java:117) ~[?:?]         at com.vmware.identity.idm.server.IdentityManager.authenticate(IdentityManager.java:9765) ~[?:?]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]         at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]         at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324) ~[?:1.8.0_101]         at sun.rmi.transport.Transport$1.run(Transport.java:200) ~[?:1.8.0_101]         at sun.rmi.transport.Transport$1.run(Transport.java:197) ~[?:1.8.0_101]         at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]         at sun.rmi.transport.Transport.serviceCall(Transport.java:196) ~[?:1.8.0_101]         at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) ~[?:1.8.0_101]         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) ~[?:1.8.0_101]         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) ~[?:1.8.0_101]         at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_101]         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) ~[?:1.8.0_101]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101]         at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]         at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276) ~[?:1.8.0_101]         at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253) ~[?:1.8.0_101]         at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162) ~[?:1.8.0_101]         at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227) ~[?:1.8.0_101]         at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179) ~[?:1.8.0_101]         at com.sun.proxy.$Proxy288.authenticate(Unknown Source) ~[?:?]         at com.vmware.identity.idm.client.CasIdmClient.authenticate(CasIdmClient.java:1274) ~[vmware-identity-idm-client.jar:?]         at com.vmware.identity.samlservice.impl.CasIdmAccessor.authenticate(CasIdmAccessor.java:467) [websso.jar:?]         at com.vmware.identity.samlservice.impl.AuthnRequestStatePasswordAuthenticationFilter.authenticate(AuthnRequestStatePasswordAuthenticationFilter.java:95) [websso.jar:?]         at com.vmware.identity.samlservice.impl.AuthnRequestStatePasswordAuthenticationFilter.authenticate(AuthnRequestStatePasswordAuthenticationFilter.java:45) [websso.jar:?]         at com.vmware.identity.samlservice.impl.AuthnRequestStateCookieWrapper.authenticate(AuthnRequestStateCookieWrapper.java:144) [websso.jar:?]         at com.vmware.identity.samlservice.impl.AuthnRequestStateCookieWrapper.authenticate(AuthnRequestStateCookieWrapper.java:40) [websso.jar:?]         at com.vmware.identity.samlservice.AuthnRequestState.authenticate(AuthnRequestState.java:461) [websso.jar:?]         at com.vmware.identity.BaseSsoController.processSsoRequest(BaseSsoController.java:82) [websso.jar:?]         at com.vmware.identity.SsoController.sso(SsoController.java:100) [websso.jar:?]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_101]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_101]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_101]         at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_101]         at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) [spring-web-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) [spring-web-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:689) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:938) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:870) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:863) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) [servlet-api.jar:?]         at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837) [spring-webmvc-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) [servlet-api.jar:?]         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292) [catalina.jar:?]         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:?]         at com.vmware.identity.SecurityRequestWrapperFilter.doFilterInternal(SecurityRequestWrapperFilter.java:49) [websso.jar:?]         at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.0.6.RELEASE.jar:4.0.6.RELEASE]         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [catalina.jar:?]         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:?]         at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-websocket.jar:8.0.33.A]         at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240) [catalina.jar:?]         at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207) [catalina.jar:?]         at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212) [catalina.jar:?]         at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) [catalina.jar:?]         at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502) [catalina.jar:?]         at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141) [catalina.jar:?]         at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) [catalina.jar:?]         at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616) [catalina.jar:?]         at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) [catalina.jar:?]         at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522) [catalina.jar:?]         at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095) [tomcat-coyote.jar:8.0.33.A]         at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672) [tomcat-coyote.jar:8.0.33.A]         at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:277) [tomcat-coyote.jar:8.0.33.A]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]         at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-util.jar:8.0.33.A]         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101] [2020-03-31T10:58:40.847Z  tomcat-http--6  eb3feb1d-8ce1-4273-b0cf-8bc7076e9396 ERROR com.vmware.identity.samlservice.AuthnRequestState] Caught Saml Service Exception from authenticate com.vmware.identity.samlservice.SamlServiceException [2020-03-31T10:58:40.847Z  tomcat-http--6  eb3feb1d-8ce1-4273-b0cf-8bc7076e9396 ERROR com.vmware.identity.BaseSsoController] Sending error to browser. ERROR: [401, Nom d'utilisateur/mot de passe incorrects], message

Any idea of what could be the problem?

0 Kudos
Cyanatide
Enthusiast
Enthusiast
‎03-31-2020 04:16 AM

Log of the previous message available in file enclosed.

websso.log.zip
0 Kudos
Cyanatide
Enthusiast
Enthusiast
‎03-31-2020 04:43 AM

I found this post:

AD authentication broken on vCenter 6.5.0.14000

Indeed, I noticed, that date of my server was sensibly changing every few minutes... I changed time synchronization to a NTP solution, it seems to be good now... I will check in the next hours/days before mark this thread as solved.

0 Kudos
berndweyand
Expert
Expert
‎03-31-2020 04:53 AM

ok - that could be a reason.

keep in mind that your vcenter-Version is 6.5 GA - over 3 years old. maybe an update helps ?

0 Kudos