VMware Cloud Community
rpotru
Enthusiast
Enthusiast

Unable to retreive health data from sps and sms - using vCSA appliance

Hi,

I have updated the vCenter 5.5 to the latest version that is available yesterday. It is linux appliance.

I was having problem to get in to web client after the update. I performed the "Regenerate SSL certificates" option thats located under Admin tab. Then the issue is resolved and I am able to get in to web client.

Now, I am seeing the different issue.

Unable to retreive health data from http://localhost/sps/health.xml

Unable to retreive health data from http://localhost/sms/health.xml

These 2 services are running but not able to load it. Also, I am seeing the following exceptions in the log.

From vpxd log:

--------------------

[7F4D4DD3A700 error 'HttpConnectionPool-000340'] [ConnectComplete] Connect failed to <cs p:00007f4cd40b2850, TCP:vc55mjl1.lab.equallogic.com:10443>; cnx: (null), error: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:

--> PeerThumbprint: 36:D5:B5:C2:8B:A5:A6:40:39:10:57:A0:0A:CF:B0:75:1B:62:31:1F

--> ExpectedThumbprint: 06:E3:CB:EB:48:12:59:47:1E:AA:05:69:B0:D5:42:14:AD:5B:63:85

--> ExpectedPeerName:xxxxxxxxxxxx

--> The remote host certificate has these problems:

-->

--> * unable to get local issuer certificate)

From vws log:

[2014-01-03 22:09:26,979 Thread-21  ERROR com.vmware.vim.health.impl.XmlUtil] Error retrieving health from url: http://localhost/sps/health.xml

java.io.IOException: Server returned HTTP response code: 503 for URL: http://localhost/sps/health.xml

        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

        at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)

        at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)

        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

        at org.apache.xerces.jaxp.validation.StreamValidatorHelper.validate(Unknown Source)

        at org.apache.xerces.jaxp.validation.ValidatorImpl.validate(Unknown Source)

        at javax.xml.validation.Validator.validate(Unknown Source)

        at com.vmware.vim.health.impl.XmlUtil.getDocumentFromUrl(XmlUtil.java:96)

        at com.vmware.vim.health.impl.ComponentSpec.retrieveHealthFromUrl(ComponentSpec.java:300)

        at com.vmware.vim.health.impl.ComponentSpec.retrieveHealth(ComponentSpec.java:266)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealthFromUrl(HealthPollerImpl.java:119)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealth(HealthPollerImpl.java:104)

        at com.vmware.vim.health.impl.HealthPollerImpl.computeHealth(HealthPollerImpl.java:203)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealth(HealthPollerImpl.java:102)

        at com.vmware.vim.health.impl.HealthPollerImpl.pollHealth(HealthPollerImpl.java:85)

        at com.vmware.vim.health.impl.HealthPollerImpl.access$100(HealthPollerImpl.java:28)

        at com.vmware.vim.health.impl.HealthPollerImpl$PollerThread.run(HealthPollerImpl.java:55)

        at java.lang.Thread.run(Unknown Source)

[2014-01-03 22:09:26,979 Thread-21  ERROR com.vmware.vim.health.impl.ComponentSpec] Unable to retrieve health for com.vmware.vim.sps from http://localhost/sps/health.xml

[2014-01-03 22:09:26,979 Thread-21  ERROR com.vmware.vim.health.impl.ComponentSpec] Unable to retrieve health for com.vmware.vim.sps from any of its health URLs

[2014-01-03 22:09:26,981 Thread-21  ERROR com.vmware.vim.health.impl.XmlUtil] Error retrieving health from url: http://localhost/sms/health.xml

java.io.IOException: Server returned HTTP response code: 503 for URL: http://localhost/sms/health.xml

        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

        at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)

        at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)

        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

        at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

        at org.apache.xerces.jaxp.validation.StreamValidatorHelper.validate(Unknown Source)

        at org.apache.xerces.jaxp.validation.ValidatorImpl.validate(Unknown Source)

        at javax.xml.validation.Validator.validate(Unknown Source)

        at com.vmware.vim.health.impl.XmlUtil.getDocumentFromUrl(XmlUtil.java:96)

        at com.vmware.vim.health.impl.ComponentSpec.retrieveHealthFromUrl(ComponentSpec.java:300)

        at com.vmware.vim.health.impl.ComponentSpec.retrieveHealth(ComponentSpec.java:266)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealthFromUrl(HealthPollerImpl.java:119)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealth(HealthPollerImpl.java:104)

        at com.vmware.vim.health.impl.HealthPollerImpl.computeHealth(HealthPollerImpl.java:203)

        at com.vmware.vim.health.impl.HealthPollerImpl.retrieveHealth(HealthPollerImpl.java:102)

        at com.vmware.vim.health.impl.HealthPollerImpl.pollHealth(HealthPollerImpl.java:85)

        at com.vmware.vim.health.impl.HealthPollerImpl.access$100(HealthPollerImpl.java:28)

        at com.vmware.vim.health.impl.HealthPollerImpl$PollerThread.run(HealthPollerImpl.java:55)

        at java.lang.Thread.run(Unknown Source)

I have loooked at the hosts file and make sure DNS is right. Also, Performed Regenerate certificates on appliance. Any ideas where to look for to troubleshoot??

Appreciated ,

Thanks!!

16 Replies
julienvarela
Commander
Commander

Hi,

Did you check this KB  : http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=200783...

Regards,

Julien

Regards, J.Varela http://vthink.fr
rpotru
Enthusiast
Enthusiast

I am not using custom ports. Also, I am using vcsa which is a linux appliance. All the settings are default. I looked at all the KB articles but none of them are useful.

Thanks for your reply.

Reply
0 Kudos
medea61
Enthusiast
Enthusiast

did you managed to get this resolved? i am stuck at the same point and banging my head against my table... Smiley Sad

Reply
0 Kudos
rpotru
Enthusiast
Enthusiast

Unfortunately, No. I saw some errors related to the SSL certificates in vCenter logs. So, We end up deploying a new vcsa which could take only few minutes as it is a test environment.

Did you regenerate SSL certificates etc? what errors are you seeing the vpxa logs? 

Try logging in to vCenter management interface. http://{vCenter name or IP here}:5480  and there is an option to regenrate SSL certs. Then restart vcenter.

See if it can resolve the issue.

Thanks,
Raj.

Reply
0 Kudos
medea61
Enthusiast
Enthusiast

Hello Raj

Wouldn't that simply replace all my carefully installed CA-signed certificates again with self-signed certificates?

Anyhow: I opened an SR for that now. I am already very curious to see how it turns out... I'll keep you posted.

Regards

Roman

Reply
0 Kudos
rpotru
Enthusiast
Enthusiast

Yes. It will replace with self signed certificates. I was working in a lab during that period.

Anyways, Let us know what you find out. It can helpful for others in future. Good luck Smiley Happy

Thanks-

Raj.

Reply
0 Kudos
bies
Contributor
Contributor

I have the same issue. Also the latest vCenter Appliance. Everything seems to work and only the  "Asset Properties History Service", VMware Profile-Driven Storage Service and "VMware vCenter Storage Monitoring Service" have this issue.

Seems a bug in the software then if more people have it.

Please send the solution here if you get one. Saves me from making another SR :smileysilly:.

Reply
0 Kudos
Fall_out
Contributor
Contributor


I also have the same issue.

Not using any custom ports, but using the vCenter application instead of the appliance.

Reply
0 Kudos
medea61
Enthusiast
Enthusiast

Hello All

So I ended up solving the issue myself. After a few hours of sleep and a little more banging of the head against the table I need to take the blame on myself for not reading and checking every little last bit of advice.

VMware explicitly states in KB 2057223 to OpenSSL v0.9.8. If you are on any recent MacOS or Linux you are most probably using OpenSSL v1.0.1 (you may check by issuing 'openssl version' on the CLI). The reason behind that is stated a few lines below the requirement of OpenSSL 0.9.8 but just not pointed out clearly: OpenSSL >1.0 uses per default PKCS#8 format for private keys. But a few services of vCenter just do not support that. Which ones you figured out already since you are reading this thread Smiley Wink

So how to resolve this issue you ask? Easy: the very same KB helps you there as well. Just issue this command to convert you private key:

openssl rsa -in pk8.key -out pk1.key


After that just follow the guidelines again to replace the certificate of the vCenter service (steps 1-12), give the appliance a reboot and you should be all set.

To be on the safe side you should convert all private key and run thru the whole guideline to replace all the certificates again


Hope this helps.


Cheers

Roman

macalb
Contributor
Contributor

Thanks, problem solved in my case "Unable to retreive health data from http://localhost/sms/health.xml"

Reply
0 Kudos
ot_o
Contributor
Contributor

hello, i have same problem. can you help me to resolve it

2014-10-13T14:16:12.285Z [7FC29F4DD700 error 'HttpConnectionPool-001515'] [ConnectComplete] Connect failed to <cs p:00007fc2a1d219c0, TCP:vcenter.local:10443>; cnx: (null), error: N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:

--> PeerThumbprint: B8:2B:6C:13:EB:E0:7B:05:DE:E3:FB:A3:E8:F3:80:47:49:84:CC:64

--> ExpectedThumbprint: 8A:46:3D:72:27:21:AB:51:4F:0F:97:B5:F1:1F:52:99:DB:BF:2F:0C

--> ExpectedPeerName: vcenter.local

--> The remote host certificate has these problems:

-->

--> * unable to get local issuer certificate)

Reply
0 Kudos
rpotru
Enthusiast
Enthusiast

Hi ot_o -

Try the following.  Login to vCenter manamgement portal https://<vCenter-IP>:5480

Go to "Admin" tab and Select Yes for "Certificate regeneration enabled".

Then Reset and reboot the vCenter appliance to apply the changes.

You should be all set afterwards. Hope it helps.

Reply
0 Kudos
ot_o
Contributor
Contributor

Hi rpotru,

I tried what you suggested, but nothing has changed.

I understand what you mean resets correctly?

I have the exact same case as you describe in the first post.

The problems started as I performed the "Regenerate SSL certificates" option that's located under Admin tab.

UKZ60IR.png

Reply
0 Kudos
ot_o
Contributor
Contributor

I saw that there was a problem Smiley Happy.

If you do not change vcenter hostname then you can not generates a new certificate And vcenter this time must be shown on the Internet

Reply
0 Kudos
INTdavhelm
Contributor
Contributor

Roman, thank you for this!

Your suggestion resolved the issue for our vCenter Server installation.  We are not running a vCSA, instead we had this exact problem on our Windows Server installation of vCenter.

The steps to resolve the problem are conceptually the same as for a vCSA -- replacing the certificates while carefully following the instruction to make sure the the private keys being used were in PKCS1 format.

It took me most of an afternoon to get the task done, and I've never been more happy to see a little bit of XML.

For others running into the problem for a Windows install of vCenter, I would recommend reading through the following KB articles:

Deploying and using the SSL Certificate Automation Tool 5.5 http://kb.vmware.com/kb/2057340

Generating certificates for use with the VMware SSL Certificate Automation Tool http://kb.vmware.com/kb/2044696


Regards,

Dave

Reply
0 Kudos
roryfeliciano
Contributor
Contributor

If you replaced the self-signed certs with CA-signed and your keys are in PCKS1 format as what Roman has mentioned, you may need to re-establish the trust between vCenter and SSO/IS. You can do this by running ssl-updater option 5 (update vCenter) then choose option 1 and option 3.

Hope this helps.

Rory

Reply
0 Kudos