Our skyline server has apparently decided to not be happy recently. I have rebooted the machine as well as restarted the endpoints which no change in status. Any insight? I have not changed anything on the server.
Here is the log file from when the screens were captured.
2018-12-17 13:32:57,329 INFO [pool-4-thread-2] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_EVENTS-odessa1v-pui.domain] Sentry is not enabled, exception ignored.
... 23 common frames omitted
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 17 common frames omitted
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 5 common frames omitted
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:106)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.doUpload(SimpleRestUploaderImpl.java:215)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.lang.Thread.run(Thread.java:748)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:139)
com.vmware.fieldtools.collector.api.exception.UploadException: Couldn't upload data to the REST endpoint.
2018-12-17 13:32:57,329 ERROR [pool-4-thread-2] VcEventCollectionTask [ExternalCollectionTask.java:198] [task=VC_EVENTS-odessa1v-pui.domain] Couldn't upload data. This upload won't be retried. Collection will continue and so this particular failed collection upload will be lost.
2018-12-17 13:32:57,100 INFO [pool-4-thread-2] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:66] [task=VC_EVENTS-odessa1v-pui.domain] Successfully collected data.
2018-12-17 13:32:55,780 INFO [pool-4-thread-2] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:59] [task=VC_EVENTS-odessa1v-pui.domain] Polling event data from VC odessa1v-pui.domain
2018-12-17 13:32:47,102 INFO [pool-4-thread-8] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_EVENTS-harris1v-pui.domain] Sentry is not enabled, exception ignored.
... 23 common frames omitted
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 17 common frames omitted
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 5 common frames omitted
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:106)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.doUpload(SimpleRestUploaderImpl.java:215)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.lang.Thread.run(Thread.java:748)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:139)
com.vmware.fieldtools.collector.api.exception.UploadException: Couldn't upload data to the REST endpoint.
2018-12-17 13:32:47,101 ERROR [pool-4-thread-8] VcEventCollectionTask [ExternalCollectionTask.java:198] [task=VC_EVENTS-harris1v-pui.domain] Couldn't upload data. This upload won't be retried. Collection will continue and so this particular failed collection upload will be lost.
2018-12-17 13:32:46,877 INFO [pool-4-thread-8] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:66] [task=VC_EVENTS-harris1v-pui.domain] Successfully collected data.
2018-12-17 13:32:45,804 INFO [pool-4-thread-8] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:59] [task=VC_EVENTS-harris1v-pui.domain] Polling event data from VC harris1v-pui.domain
2018-12-17 13:32:25,755 INFO [pool-4-thread-1] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_EVENTS-odessa1v-pui.domain] Sentry is not enabled, exception ignored.
... 23 common frames omitted
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 17 common frames omitted
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 5 common frames omitted
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:106)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.doUpload(SimpleRestUploaderImpl.java:215)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.lang.Thread.run(Thread.java:748)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:139)
com.vmware.fieldtools.collector.api.exception.UploadException: Couldn't upload data to the REST endpoint.
2018-12-17 13:32:25,755 ERROR [pool-4-thread-1] VcEventCollectionTask [ExternalCollectionTask.java:198] [task=VC_EVENTS-odessa1v-pui.domain] Couldn't upload data. This upload won't be retried. Collection will continue and so this particular failed collection upload will be lost.
2018-12-17 13:32:25,520 INFO [pool-4-thread-1] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:66] [task=VC_EVENTS-odessa1v-pui.domain] Successfully collected data.
2018-12-17 13:32:24,361 INFO [pool-4-thread-1] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:59] [task=VC_EVENTS-odessa1v-pui.domain] Polling event data from VC odessa1v-pui.domain
2018-12-17 13:32:15,791 INFO [pool-4-thread-6] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=VC_EVENTS-harris1v-pui.domain] Sentry is not enabled, exception ignored.
... 23 common frames omitted
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 17 common frames omitted
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 5 common frames omitted
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:106)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.doUpload(SimpleRestUploaderImpl.java:215)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.lang.Thread.run(Thread.java:748)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:139)
com.vmware.fieldtools.collector.api.exception.UploadException: Couldn't upload data to the REST endpoint.
2018-12-17 13:32:15,791 ERROR [pool-4-thread-6] VcEventCollectionTask [ExternalCollectionTask.java:198] [task=VC_EVENTS-harris1v-pui.domain] Couldn't upload data. This upload won't be retried. Collection will continue and so this particular failed collection upload will be lost.
2018-12-17 13:32:15,569 INFO [pool-4-thread-6] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:66] [task=VC_EVENTS-harris1v-pui.domain] Successfully collected data.
2018-12-17 13:32:14,495 INFO [pool-4-thread-6] c.v.f.c.c.VcEventCollectionTask [VcEventCollectionTask.java:59] [task=VC_EVENTS-harris1v-pui.domain] Polling event data from VC harris1v-pui.domain
2018-12-17 13:32:11,733 INFO [pool-4-thread-5] c.v.s.c.s.SkylineSentry [SubstituteLogger.java:169] [task=heartbeat] Sentry is not enabled, exception ignored.
... 23 common frames omitted
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 17 common frames omitted
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
... 5 common frames omitted
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.lambda$uploadData$0(SimpleRestUploaderImpl.java:106)
at com.vmware.fieldtools.collector.common.upload.rest.SimpleRestUploaderImpl.doUpload(SimpleRestUploaderImpl.java:215)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
Hello,
Thank you for sharing the details,
As per the screen shots shared, We see that the VC_HOST, VC_ EVENTS, VC_CHANGES Endpoints are all having errors and Status is not working.
Could you please check the collector version which is currently installed.
Importing the certificate(s)/chain into the collector manually by following the steps mentioned in the article below:
"Skyline Collector registration fails with error: "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
Hello,
Thank you for sharing the details,
As per the screen shots shared, We see that the VC_HOST, VC_ EVENTS, VC_CHANGES Endpoints are all having errors and Status is not working.
Could you please check the collector version which is currently installed.
Importing the certificate(s)/chain into the collector manually by following the steps mentioned in the article below:
"Skyline Collector registration fails with error: "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
I know the root password, but it is not taking it when I try to connect with SSH.
By default, SSH is disabled. To enable SSH access, perform the following steps:
While using network rules that modify the certificate/certificate chain when attempting to connect to https://vcsa.vmware.com.
If there is a certificate chain in use we will need to update the certificate in the Java Keystore.
Page not found, can you post a new valid link to the KB?