VMware Cloud Community
drunkenbloke
Contributor
Contributor
Jump to solution

Small oddity when updating certificates, I think I have missed a root chain out somewhere.

Hi all,

We are running vmware 6.5 and pretty much all of the certificates are due to expire over the next few weeks and the person that has set everything up has now left so I cannot ask how this was set up.

So far I have gone through the following on our sandpit environment,

On the PSC server

Use certificate-manager menu option 2 to generate a new cert request.

Run the request through the ca server.

Import the new cert to the PSC server.

From the PSC web gui log into certificate management for the PSC and renew the __MACHINE_CERT, machine and vpshere-webclient certs.

Log out certificate management.

Log certificate management into the vcenter appliance.

Use the GUI to renew __MACHINE_CERT, machine and vsphere-webclient, vpxd and vpxd-extension certs.

I have also updated /etc/applmgmt/appliance/new.cert and the config and restarted lighthttp.

At this point everything looks to be OK most of the websites have the full cert chain if you view the certificates. For some reason the vcenter appliance website cert does not display the chain. If I download the certs from the login page and install them then this fixes the chain. Should I have to do this on every machine that logs in or should the chain be included as it is on the PSC webpage login.

Also on our production system if you log certificate management into a vcenter appliance and view the solution user certs you can see the chains but these are not showing on the sandpit. This leads me to think I have missed loading a chain somewhere but cannot see it missing from anywhere.

This is how the cert appears in the production SPC certificate manager.

pastedImage_1.png

And this is how it looks on the sandpit SPC certificate manager.

pastedImage_2.png

Can anyone advise what I might of missed out ?

1 Solution

Accepted Solutions
Virtbay
Enthusiast
Enthusiast
Jump to solution

Yes you said it right GUI is missing something. when it comes to certificate replacement at-least in this version (seeing the certificate management GUI only from this version).GUI is something which I would not go for. I think from vcenter 7.0 it should be much more effective via GUI to replace certificates.

Regards VB Please mark helpful or correct if my answer resolved your issue.

View solution in original post

Reply
0 Kudos
5 Replies
Virtbay
Enthusiast
Enthusiast
Jump to solution

Hi Drunkenbloke,

I see that you have replaced the vmca root certificate (VMCA as subordinate CA). How does the chain look like if we go to the Vcenter URL or PSC URL and view it via web browser rather than the certificate management page ?

Regards VB Please mark helpful or correct if my answer resolved your issue.
Reply
0 Kudos
drunkenbloke
Contributor
Contributor
Jump to solution

Hi Virtbay

On the PSC it looks correct with the full chain.

pastedImage_0.png

On the vcenter this chain is missing and only showing the cert for the machine. If I download and install the trusted root certs then it looks correct.

Without downloading the root ca certificates

pastedImage_1.png

With them downloaded.

pastedImage_2.png

In both cases it also states that the certificate is invalid.

Reply
0 Kudos
Virtbay
Enthusiast
Enthusiast
Jump to solution

Hi

That is odd as you had mentioned, To be honest the webUI certificate management has never helped me out and given that it is 6.5 it could be that the certificate did not replace properly while running through WebUI or it could be also due to the fact the certs are not pushed to the roots in your environment but it is kind of odd that it displays the chain for PSC but not for VC. Do run the replacement again on the Vcenter appliance by using the certificate-manager script. Run through  option 3 and 6 (replace machine SSL and solution user with VMCA certificate) and see whether that helps out.Do take the snapshot of the VC before performing any changes on the Vcenter.

Regards VB Please mark helpful or correct if my answer resolved your issue.
Reply
0 Kudos
drunkenbloke
Contributor
Contributor
Jump to solution

Hi Virtbay

Sorry this is a bit of a rambling reply I have been writing it as I have been looking at the problem.

As I was reading a guide I had not even thought about trying from the certificate manager on the vcenter.

I have gone through and options 3, 6 and 8 and they all error Smiley Sad

On the bright side there is at least an error message.

2020-03-12T23:37:16.302Z ERROR certificate-manager 'lstool get' failed: 1

I have checked and all of the appliances are 6.5 U2 which is supposed to fix this.

The service causing the problem is a netapp one.

Get service 120ba3fd-a6d4-4fc9-a341-e037d08cd2a1_com.netapp.scvm.webclient

Status : 0% Completed [Reset operation failed]

Even though this is not completing the __MACHINE_CERT is now showing some of the root chain so some of the way there.

After poking around the sandpit environment some more and it looks like the netapp appliances and servers have been completely removed so I have un registered all of the netapp extensions and had another try. Everything is looking successful except the vmonapi service is not starting which then causes everything to try to rollback.

If I catch the web login before the roll back happens then the vcenter web page cert is now showing some of the root chain (Not the Root CA but the rest like the __MACHINE_CERT above) so I have to fix that service next.

Having had a look at the logs I have found this KB

VMware Knowledge Base

Adding the PSC to the NO_PROXY config has that service fixed and the certification_manager process now runs through.

So now that everything works without any errors I have gone through the whole process from PSC and VC using just the cli and now think we might be about there.

When using the PSC certification manager all certs on the PSC and VC now show back to the CA server (Just missing the Root for some reason)

pastedImage_8.pngpastedImage_9.png

But when going to the web page SP Root CA is there

pastedImage_10.png

Which is just how things appear in our production systems so I think the answer here is to use the CLI as the gui method is missing something along the way.

Cheers for the assistance Virtbay.

Virtbay
Enthusiast
Enthusiast
Jump to solution

Yes you said it right GUI is missing something. when it comes to certificate replacement at-least in this version (seeing the certificate management GUI only from this version).GUI is something which I would not go for. I think from vcenter 7.0 it should be much more effective via GUI to replace certificates.

Regards VB Please mark helpful or correct if my answer resolved your issue.
Reply
0 Kudos