Highlighted
Contributor
Contributor

Installing CA-signed SSL certificates on vCenter 5.1 (Server or Appliance)

Jump to solution

I recently upgraded my ESXi 5.0 hosts to ESXi 5.1 and they all kept the CA-signed SSL certificates I previously installed. I did a fresh install of vCenter 5.1 Server where the same box ran SSO, Inventory Services, vCenter Server, and Update Manager. After install, everything was working perfectly except that none of the vCenter services were using my CA-signed SSL certificate - only the ESXi 5.1 hosts had these.

So I followed the directions in the Replacing Default vCenter 5.1 and ESXi Certificates PDF found at http://www.vmware.com/resources/techresources/10318. The document is terrible. For example, page 10 lists the three default locations for SSL certificates on Windows 2008. None of these paths are correct. The first has a typo of an extra space between "Program" and "Data" and the other two say "Program Files" when they should have been "ProgramData". This is just the beginning of the problems.

If you follow the directions to the letter, you'll break vCenter. I got frustrated and thought I'd give the vCenter 5.1 Appliance a shot. With respect to CA-signed SSL certificates, it was worse. The vCenter 5.1 Appliance can't even auto-generate a new SSL certificate if you change the hostname (turn on auto-certificate generation, change hostname and reboot). It gives a 653 error during the boot up process and keeps the original certificate. Don't even bother trying the steps on page 18 in the aforementioned guide - you'll just get the same 653 error.

It seems to me that VMware has not done any testing around CA-signed SSL certificate installation on vCenter 5.1. It's amazing to me that SSL certificate installation is so tedious for vCenter and ESXi when vShield Manager 5.1 has a very simple process that works well (and is similar to the SSL certificate installation process on DRACs, RSAs, iLOs, various firewalls, etc.).

I did a lot of Google searches and found various blogs on SSL certificate installation but many were based on pre-5.1 GA products. If you have had any success installing CA-signed SSL certificates with vCenter Server or Appliance 5.1 GA, please let me know how you got around some of these issues. Please indicate whether your vCenter Server or Appliance was running on an ESXi 5.1 GA host as well. Please don't respond regarding vCenter 5.0 - I didn't have any issues with it and SSL certificates (other than that it was more tedious that it had to be).

Thanks in advance,

Nate

1 Solution

Accepted Solutions
Highlighted
Contributor
Contributor

In the end i've managed to get it to install by giving it 127.0.0.1 rather than the externally accessible IP of the vCenter server, this is fine in my case as the vCenter server and VUM are on the same VM but its not exactly ideal for bigger deployments.

View solution in original post

0 Kudos
28 Replies
Highlighted
Enthusiast
Enthusiast

Hi natedev,

Appreciate effort and self exploration skill.

Could you please upload the auth.log file from mentioned path /var/log/auth.log.

"concentrate the mind on the present moment."
0 Kudos
Highlighted
Contributor
Contributor

Hi natedev,

ANy chance you managed to replace the certs I have tried both pre-install (caused install to fail complaining about already being registered) and post install following the same document that broke my vCenter install.

I also noticed the mistakes with the file locations and also found some issues where it referenced scripts that don't exist (unsure if that is just my install).

Any help would be great or better yet a proper guide written in the order that is needed to complete the process!

0 Kudos
Highlighted
Contributor
Contributor

Hi to get the trusted certs to work each service needs a different cert and all the distinguished name fields need to be unique not just the common name.

I just generate certs with only a common name in the dn section appart from the one for vcenter service as it needs the other values. On each cert I set the common name to be hostname.domain.sufix-service name on the one for web client just have the fqdn of the host as browsers like that but it's not essential.

Also web client needs certs in place before install as the scripts for post registration after cert change don't exist. Even better have all certs in place before the install accept I have not worked out how to do this for sso

Then just remember to add all certs and ca certs to the trust store in programdata\vmware\ssl rename the files to {hash}.0 {hash}.1 etc to get the hash for each cert OpenSSL x509 -subject_hash_old -noout -in rui.crt it's a good idea to have these in place before the install as then you know it all works if the installer does not prompt to install a cert into the trust store.

If you need more extensive instructions let me know I'm just on an iPad at the moment so not that easy to type extensive instructions.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Yes I've been trying to follow the SSL certificate guide myself and it is a terrible document. To help others out I've written a series of blog posts, may of which are focused on SSL certificate replacement steps. They are based on the VMware guide, but fixed their mistakes and added a lot of screenshots. That's not to say the process is error free...I'm still getting inventory service registration errors and have an open ticket with VMware. But maybe they will help you out:

http://derek858.blogspot.com/2012/09/vmware-vcenter-51-installation-part-1.html

Derek Seaman
0 Kudos
Highlighted
Contributor
Contributor

I managed to get it working by making each cert have a different OU field and generating the key using an old rsa command.

So if my vCenter was Server.Domain.local the openssl config file is like below for each certificate. I am still testing to see what fields can be used on each cert but it takes time to test each scenario.

All the keys need to be generated in RSA format using: c:\OpenSSL-Win32\bin\openssl genrsa -out rui.key 2048

then the certificate signing request generated using: c:\OpenSSL-Win32\bin\openssl req -out rui.csr -key rui.key -new -config vcenter.cfg

****************************************************************************************************************
[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS:Server, DNS:Server.Domain.local
[ req_distinguished_name ]
countryName = NZ
stateOrProvinceName = Auckland
localityName = Auckland
0.organizationName = Company Ltd.
organizationalUnitName = Put a unique name here for each cert like {Inventory Service}
commonName = Server.Domain.local
****************************************************************************************************************

Also change each cert as you install each service like Derek lays out in his blog. For Web-Client and Log Browser you need to put them in the correct place before instalation as the register bach files don't unregister the old ones and won't update web-client corectly.

Alternativly put the certs in place before install accept SSO will need to be changed manually after it has been installed before installing the next service.

Also remember to place a copy of all CA certs (root and Subordinate) into ProgramData\VMware\SSL name them hash.0 hash.1 hash.2 e.g

CA = {hash}.0 Subordinate CA = {hash}.1

the hash can be found in openssl using "openssl x509 -subject_hash_old -noout -in rui.crt" on each of the CA certs. also save here a file called ca_certificates.crt that is the PEM certificate file for your root CA.

After this I had a full install of vCenter with each compnent using a trusted certificate. however Log browser does not work (see below posts). errors about not being able to find the trust chain for the certificate web-client is providing.

Orchestrator is alot more involved; you have to generate and import all certs for Orchestrator into the keystore this is a bit more involved and I have only dont it on the Appliance but the Windows install is not much different. VMware doc's are nto that great but basically tou remove all certs in the java keystore all passwords are dunesdunes and then import your chain into it and give the certificate you want it to use the alias dunes. there are two keystores but one is a symbolic link to the other so only update one of them. Then if you are using the appliance you can also replace the appliance config one in the lighttpd directory with a new pem file not password protected.

Any questions let me know. I will be helping Derek put all this into his blog once it is all confirmed working for easy to follow instructions.

Message was edited by: Terafirma - Final tested setup and confirmed working. Log Browser error still exists.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Terafirma, thanks for the detailed information! I'll try it out in my lab, and if I get success then I'll update my blog posts. This is so much harder than it needs to be!

Derek Seaman
Highlighted
Contributor
Contributor

Also missed one last step is to copy the certificate files for the Web-Client to the log browser directory:

C:\Program Files\VMware\Infrastructure\vSphereWebClient\logbrowser\conf\

also the log browser refrences the SSO cert of:

%PROGRAMDATA%/VMware/SSL/ca_certificates.crt so make sure you have a copy of the SSO cert in there with that name.

Also found the register script for web-client is called client-repoint.bat (not register-sso.bat like VMware doc says) this scrit does not unregister the existing service tho so not sure how well it work as I put my certs in place first.

0 Kudos
Highlighted
Contributor
Contributor

Sorry just checked and my log browser give me this error:

Capture.JPG

If I reinstall with no ssl certificates for logbrowser I could get things going with the VMware cert. so not sure what this error is in relation to. certainly hope it is not because that service does not like my complex password.

0 Kudos
Highlighted
Contributor
Contributor

Have tried and not managed to get the log browser serivce working.

First problem is it lookes for the ca_certificates.crt in programdata\vmware\SSL and expects it to be in DER format its not (what engineer wrote that code) so I put a copy of the SSO trust chain into the logbrowser config directory and edit its properties file to change it to that one.

That fixed the first error in the log of not being able to load the file now it says that it has loaded the file in and found the 3 certificates (Root CA, Intermediate CA and SSO cert).

Then when the service tries to do anything it gets an error as shown below:

2012-09-21 10:54:00,792 INFO [com.vmware.vide.ws.sso.SsoRequestHandler] [WrapperStartStopAppMain] Using certificate file ../conf/sso_certificates.p7b
2012-09-21 10:54:00,808 INFO [com.vmware.vide.ws.sso.SsoRequestHandler] [WrapperStartStopAppMain] Loaded 3 certificate(s)
2012-09-21 10:54:01,276 INFO [org.apache.cxf.endpoint.ServerImpl] [WrapperStartStopAppMain] Setting the server's publish address to be http://0.0.0.0:12221/
2012-09-21 10:54:01,323 INFO [org.apache.cxf.jaxrs.provider.ProviderFactory] [WrapperStartStopAppMain] Problem with setting the default provider org.apache.cxf.jaxrs.provider.JSONProviderorg/codehaus/jettison/mapped/TypeConverter
2012-09-21 10:54:01,354 INFO [org.apache.cxf.jaxrs.provider.ProviderFactory] [WrapperStartStopAppMain] Problem with setting the default provider org.apache.cxf.jaxrs.provider.JSONProviderorg/codehaus/jettison/mapped/TypeConverter
2012-09-21 10:54:01,385 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] jetty-7.3.1.v20110307
2012-09-21 10:54:01,448 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] Started SelectChannelConnector@0.0.0.0:12221
2012-09-21 10:54:01,479 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] started o.e.j.s.h.ContextHandler{,null}
2012-09-21 10:54:01,641 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] stopped o.e.j.s.h.ContextHandler{,null}
2012-09-21 10:54:01,703 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] jetty-7.3.1.v20110307
2012-09-21 10:54:01,703 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] started o.e.j.s.h.ContextHandler{,null}
2012-09-21 10:54:01,813 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] NO JSP Support for /logbrowser-webclient, did not find org.apache.jasper.servlet.JspServlet
2012-09-21 10:54:01,844 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] started o.e.j.w.WebAppContext{/logbrowser-webclient,file:/C:/Program%20Files/VMware/Infrastructure/vSphereWebClient/logbrowser/workspace/public/},C:\Program Files\VMware\Infrastructure\vSphereWebClient\logbrowser\workspace\public
2012-09-21 10:54:02,015 INFO [org.eclipse.jetty.util.log] [WrapperStartStopAppMain] Started SslSocketConnector@0.0.0.0:12443
2012-09-21 11:05:11,428 INFO [com.vmware.vim.sso.client.impl.X509TrustChainKeySelector] [qtp1169368970-39] Failed to find trusted path to signing certificate <CN=Server.Domain.local>
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
****************************************************************************************************************************************************************************
All I can think is that the software opens the file in DER format but then the service tries to use the certificates in X509 format and thus cannot find the trust chain. I am now looking to see if there is a way to tell the software to read in an X509 format file.
Might have to do a blank install of vCenter to see how it runs on their certs. If anyone has any ideas any help woulld be apreciated.
Seems to be related to the pfx file that the service uses not having the full certificate chain in it.

Message was edited by: Terafirma

0 Kudos
Highlighted
Contributor
Contributor

Further testing has shown that each certificate only needs to have a unique OU property all the other values can be the same. I have not fully tested it but have been through all the logs and default installs and compared everything along with what it is doing in the database and only found this value to matter.

0 Kudos
Highlighted
Contributor
Contributor

Terafirma, are you able to install VUM? As following what you have said as regards to certificates I have managed to sucessfully install SSO, IS and vCenter but upon entering the details for VUM to logon to the vCenter server the installer hangs.

Further research shows that the vciinstallutils.exe process where it checks the vCenter version never completes and goes round in circles.

0 Kudos
Highlighted
Contributor
Contributor

Hi yozza_uk

Yes I am running vC Update Manager with a trusted cert. I had the problem you are having once and managed to fix it by supplying the ip address of the vcenter server to logon to as the config process was getting the link local ipv6 address and didn't like it. The easiest way for vC UM is to change it afterwards as it is an easy one to do. currently the only product with a gui tool to do it.

Also I have done some further digging and found that what I said above is out and only one field needs to be unique. (I have looked through so much vCenter code and config files its not funny) I have managed to narrow it down to the OU field and I think VMware chose that one as it it is one of the few that can be unique even with online certificate providers.

I am now testing this out so not 100% on it yet but will update the post above with the details.

Also have found that some of the services want their certificates (pfx files) witht he full chain so just working on that part now as well.

Will contemplate writing a script to do the whole lot automatically once I have nailed down the correct setup for each service.

0 Kudos
Highlighted
Contributor
Contributor

Unfortunately i'm already using the IP address with the same result. Nothing I have tried seems to effect it in any way, if I use the wrong credentials then it will fail and it will also fail using 127.0.0.1 to connect as it isn't listed on the SSL certificate. This is multiply installations all giving the same results, the only time i've got it to work is a brand new default install with the auto generated self signed certs in place.

Auto Deploy and the Syslog Collector both install and register fine.

0 Kudos
Highlighted
Contributor
Contributor

In the end i've managed to get it to install by giving it 127.0.0.1 rather than the externally accessible IP of the vCenter server, this is fine in my case as the vCenter server and VUM are on the same VM but its not exactly ideal for bigger deployments.

View solution in original post

0 Kudos
Highlighted
Enthusiast
Enthusiast

Has anyone managed to change the certs on the vCenter Server Appliance? When I enter the command from the VMware pdf, '/usr/sbin/vpxd_servicecfg certificate change rui.crt rui.key', I keep getting an error code of 659, which is 'ERROR_CERTIFICATE_MISTERY'. I've tried putting the CA root cert in /etc/certs/ssl, with a symlink for it at /etc/certs/ssl/hash.0. I've tried importing the root cert into the java keystore at /usr/java/jre-vmware/lib/security/cacert (password is changeit). I believe my cert is trusted, because I can run 'openssl verify rui.crt' and it comes back OK. I can't figure out for the life of me what the error code means.

0 Kudos
Highlighted
Enthusiast
Enthusiast

Alright, I figured it out. Had to concatenate the server cert and the root cert into rui.crt, then pass that and the server key to the vpxd_servicecfg command.

Assuming the certs are named server.crt and root.crt, and the server key is already rui.key, the steps below should work.

1. cat server.crt root.crt > rui.crt

2. /usr/sbin/vpxd_servicecfg certificate change rui.crt rui.key

Success message is: VC_CFG_RESULT=0

Now to get everything working behind a load balancer...

Cheers,

-Loren

Highlighted
Contributor
Contributor

lorengordon's post about concatenating the CA certificate with the host certificate looks promising for resolving the issues related to the vCenter 5.1 Appliance.

Many others have made some good suggestions regarding installing the certificates on the various components for full-blown vCenter and the various components.

0 Kudos
Highlighted
Enthusiast
Enthusiast

I'm building a script to configure the VCSA and change out the certs. I'll post it once I've done a bit more testing...

Highlighted
Enthusiast
Enthusiast

Attached is a script that should change out the certificates on the VCSA. It also does a number of other things that you may or may not want. It is pretty heavily commented so you can always comment out sections if you want.

For example, I plan to use a Windows vCenter Server with the Inventory service installed locally, so this script disables those services in the VCSA. I've left the SSO and vSphere Web Client enabled, along with the log browser, netdumper, and syslog-collector.

There is a fair bit of error checking, but use at your own risk. Study the script and test it in a non-production environment.

#CHANGELOG - Version - 2012.10.08-01

Here's the features from the script header:

#FEATURES - this script will

# 1. Accept the EULA
# 2. Configure the fully-qualified hostname
# 3. Join Active Directory (if desired)
# 4. Enable the embedded vCenter SSO service
# 5. Replace the self-signed SSL certs with those provided by a CA
# 6. Re-configure the vSphere Web Client and SSO service to function behind a load-balancer
# 7. Disable undesired embedded services (chkconfig <service> off): vcenter, inventory (for use with an external vCenter Server)
# 8. Set the SSO master password and the admin password
# 9. Add the AD domain as an SSO identity source
#10. Set the root password

Let me know if you have any issues or suggestions.

Cheers,

-Loren

0 Kudos