VMware Cloud Community
HariRajan
Hot Shot
Hot Shot

VRA 7 strange failure

all services are failed on the working setup , any idea ?

2016-04-11 21:48:21,542 INFO (SimpleAsyncTaskExecutor-1) [;;] com.vmware.horizon.connector.utils.RestClient - END   sendRequestBase (https://xxxxxxx/SAAS/t/VSPHERE.LOCAL/jersey/manager/api/health, ..., null, GET, null

, ...)

2016-04-11 21:48:21,542 ERROR (SimpleAsyncTaskExecutor-1) [;;] com.vmware.horizon.pullengine.HealthCheckClient - Error while checking the health of horizon service

2016-04-11 21:48:21,542 INFO (SimpleAsyncTaskExecutor-1) [;;] com.vmware.horizon.connector.utils.ServiceUtils - Waiting 30 seconds to check again. Attempts = 93

2016-04-11 21:48:31,966 INFO (SimpleAsyncTaskExecutor-2) [;;] com.vmware.horizon.connector.utils.ServiceUtils - Checking health of service for tenant VSPHERE.LOCAL

2016-04-11 21:48:31,966 INFO (SimpleAsyncTaskExecutor-2) [;;] com.vmware.horizon.connector.utils.RestClient - BEGIN sendRequestBase (https://ixxxxxx/SAAS/t/VSPHERE.LOCAL/jersey/manager/api/health, ..., null, GET, null

2016-04-11 21:56:22,911 INFO (Timer-15) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.client.rest.Utils - END   sendRequestBase (https:/xxxxxx/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token, ..., application/x-www-form-urlen

coded, POST, null, ...)

2016-04-11 21:56:22,912 INFO  (Timer-15) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.client.rest.IApi - Failed to getOAuth2AccessToken: https://ixxxxx/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token : 404

2016-04-11 21:56:22,912 WARN  (Timer-15) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.connector.admin.service.BrandingScheduleService - Syncing branding configuration from Horizon failed. Continuing to use existing branding information.

2016-04-11 21:56:22,920 INFO (Timer-16) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.connector.admin.service.BrandingScheduleService - Syncing password recovery information

2016-04-11 21:56:22,920 INFO (Timer-16) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.client.rest.Utils - BEGIN sendRequestBase (https://ixxxx/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token, ..., application/x-www-form-urlen

coded, POST, null, ...)

2016-04-11 21:56:22,923 INFO (Timer-16) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.client.rest.Utils - END   sendRequestBase (https://xxx/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token, ..., application/x-www-form-urlen

coded, POST, null, ...)

2016-04-11 21:56:22,923 INFO (Timer-16) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.client.rest.IApi - Failed to getOAuth2AccessToken: https://ie3xxxxm/SAAS/t/VSPHERE.LOCAL/API/1.0/oauth2/token : 404

2016-04-11 21:56:22,924 WARN  (Timer-16) [3001@VSPHERE.LOCAL;;] com.vmware.horizon.connector.admin.service.BrandingScheduleService - Syncing password recovery configuration from Horizon failed. Continuing to use existing password recover

ixxxx0:/var/log/vmware/horizon #

  1. access.log
  2. 10.78.185.105 [12/Apr/2016:22:16:50 +0530][1 ms] "POST /SAAS/t/vsphere.local/auth/oauthtoken?grant_type=client_credentials HTTP/1.1" 404 1947 [tomcat-http--7]
  3. 10.78.185.105 [12/Apr/2016:22:16:50 +0530][1 ms] "POST /SAAS/t/vsphere.local/auth/oauthtoken?grant_type=client_credentials HTTP/1.1" 404 1947 [tomcat-http--16]
  4. 10.78.185.105 [12/Apr/2016:22:16:51 +0530][1 ms] "POST /SAAS/t/vsphere.local/auth/oauthtoken?grant_type=client_credentials HTTP/1.1" 404 1947 [tomcat-http--50]
  5. 10.78.185.105 [12/Apr/2016:22:17:00 +0530][1 ms] "POST /SAAS/t/vsphere.local/auth/oauthtoken?grant_type=client_credentials HTTP/1.1" 404 1947 [tomcat-http--4]
  6. 10.78.185.105 [12/Apr/2016:22:17:00 +0530][1 ms] "POST /SAAS/t/vsphere.local/auth/oauthtoken?grant_type=client_credentials HTTP/1.1" 404 1947 [tomcat-http--44]

  1. Catalina.out

                <div class="footer-text">

                    Identity Manager 2.5.0.0 Build 3235845. Copyright &copy; 2013-2015 VMware, Inc. All rights reserved. This product is protected by copyright and intellectual property laws in the United States and other countries as well as by international treaties. VMware products are covered by one or more patents listed at <a href="http://www.vmware.com/go/patents">http://www.vmware.com/go/patents</a>.

                </div>

            </div>

        </div>

    </body>

</html>

        at com.vmware.vcac.authentication.oauth.support.OAuthAccessTokenRetriever.contactHorizonAuthenticationEndpoint(OAuthAccessTokenRetriever.java:253) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.oauth.support.OAuthAccessTokenRetriever.getAccessToken(OAuthAccessTokenRetriever.java:157) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.oauth.support.OAuthAccessTokenRetriever.getAccessTokenByClientIdAndSecret(OAuthAccessTokenRetriever.java:66) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.sts.OAuth2TokenService$SolutionTokensHolder.newValue(OAuth2TokenService.java:179) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.sts.OAuth2TokenService$SolutionTokensHolder.newValue(OAuth2TokenService.java:166) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.platform.support.TemporalVariable.get(TemporalVariable.java:40) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.sts.OAuth2TokenService.acquireSolutionToken(OAuth2TokenService.java:57) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.authentication.sts.OAuth2TokenService$SolutionOAuth2Authentication.getOAuthToken(OAuth2TokenService.java:267) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.platform.rest.client.authentication.oauth2.BaseOAuth2Authentication.hashCode(BaseOAuth2Authentication.java:43) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.platform.rest.client.authentication.oauth2.OAuth2AccessTokenAuthentication.hashCode(OAuth2AccessTokenAuthentication.java:58) ~[platform-security-sso-client-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.componentregistry.rest.client.RestClientEndPointFactory.getComponentRegistryRestClient(RestClientEndPointFactory.java:217) ~[component-registry-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.componentregistry.rest.client.RestClientEndPointFactory.getEndPointService(RestClientEndPointFactory.java:179) ~[component-registry-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.componentregistry.rest.client.RestClientEndPointFactory.getDefaultRestClientByEndPointTypeWhenAvailable(RestClientEndPointFactory.java:267) ~[component-registry-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.componentregistry.rest.client.SolutionRestClientManager.restClientForSolutionUserByTypeWhenAvailable(SolutionRestClientManager.java:88) ~[component-registry-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.componentregistry.rest.client.SolutionRestClientManager.restClientForSolutionUserByTypeWhenAvailable(SolutionRestClientManager.java:82) ~[component-registry-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.eventlog.rest.client.service.ServiceEventPublisher.getEventLogRestClient(ServiceEventPublisher.java:223) ~[eventlog-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.eventlog.rest.client.service.ServiceEventPublisher.postEvent(ServiceEventPublisher.java:175) ~[eventlog-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.eventlog.rest.client.service.ServiceEventPublisher.postEvent(ServiceEventPublisher.java:143) ~[eventlog-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at com.vmware.vcac.core.eventlog.rest.client.service.ServiceEventPublisher$1.run(ServiceEventPublisher.java:108) [eventlog-client-rest-service-7.0.0-SNAPSHOT.jar:?]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_66]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_66]

        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_66]

Caused by: org.springframework.web.client.HttpClientErrorException: 404 Not Found

        at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91) ~[spring-web-4.2.2.RELEASE.jar:4.2.2.RELEASE]

          at

Thanks & Regards in Plenteous . Hari Rajan
Reply
0 Kudos
5 Replies
GrantOrchardVMw
Commander
Commander

Horizon is providing auth, both to users and between services. Hence all the subsequent failures.

Grant

Grant http://grantorchard.com
Reply
0 Kudos
HariRajan
Hot Shot
Hot Shot

Hi Grant,


yes  , I just got an another evidence of the issue and  SR is opened with VMware , hopefully engineering team will look this issue soon.

I have tried multiple method to connect the LDAP  and no luck since SSO service itself fails.


Regarding the certificate , I am not seeing any issues and pushed the cert again and trust is re-established ,


DNS and Name resolution is working perfectly and this issue happened all in sudden in a working environment ,


By looking below error , i just wondering did the administrator@vsphere.local is blocked / disabled !.  ,

    <div class="footer clear">

<div class="footer-inner">

<div class="footer-text">

Identity Manager 2.5.0.0 Build 3235845. Copyright &copy; 2013-2015 VMware, Inc. All rights reserved. This product is protected by copyright and intellectual property laws in the United States and other countries as well as by international treaties. VMware products are covered by one or more patents listed at <a href="http://www.vmware.com/go/patents">http://www.vmware.com/go/patents</a>.

</div>

</div>

        </div>

    </body>

</html>

  • org.springframework.security.authentication.BadCredentialsException:

referred

http://docs.spring.io/autorepo/docs/spring-security/4.0.3.RELEASE/apidocs/org/springframework/securi...

Thrown if an authentication request is rejected because the credentials are invalid. For this exception to be thrown, it means the account is neither locked nor disabled.”

Thanks & Regards in Plenteous . Hari Rajan
Reply
0 Kudos
dohko_44
Contributor
Contributor

Hi HariRajan,

I am in a similar situation after joining an appliance in a cluster. The Identity Manager seems completely broken in that appliance and prints similar errors as yours.

How did it go with VMware support? Could they fix it in the end?

Thanks.

Reply
0 Kudos
bradger33
Enthusiast
Enthusiast

Hi, i have got the exact same issue in a distributed vRA enviornment. Did you ever manage to resolve this?

thanks

Reply
0 Kudos
qc4vmware
Virtuoso
Virtuoso

Running into a similar issue in 7.5 HF6.  Did you ever find a resolution?

Reply
0 Kudos