How to fix VMware View Connection Server Certificate Revocation Check Error?

[Fab4UB]

Dear community,

Since about 2 weeks I am experiencing a certificate revocation check error in our Horizon View 6.2 environment. The strange thing is that within about 12 hours both (replicating) connection servers and the vCenter server / Composer server (on the same machine) are considered to have invalid certificates, though, in fact, they have valid ones (CA certificates). We do not use security servers.

The view administrator console shows the following for the connection servers:

Server's certificate is not trusted.

Server's certificate cannot be checked.

For the vCenter it says (as I manually validated the certificate):

No problem detected.

Certificate is untrusted but the thumbprint for the certificate is accepted.

With logging set to "full" the connection server logs show the following for the vCenter server:

TRACE (17B0-0E98) <VCHealthUpdate> [NativeKeyVault] validateCertificateChain response: { Result=FAIL, EndEntityReasons=cantCheckRevoked, ChainReasons=invalid, SelfSigned=false, EndErrorCode=16777280, EndInfoCode=258, ChainErrorCode=16777280, ChainInfoCode=256, PolicyErrorCode=-2146885613 }

As far as I can see there are no similar entries for the connection server certificates in the log.

At the moment I am running the environment with manually validated Composer and vCenter certificates and invalid (red) connection server certificates (as View clients and browsers do not consider them to be invalid).

I already checked that I am able to make everything "green" again via setting the registry key "CertificateRevocationCheckType" to 2 (as described here Configuring Certificate Revocation Checking on Server Certificates). This leads me to the conclusion that one of the intermediate certificates cannot be validated. As well, I got the information that one "version" of an intermediary certificate (of an intermediary ca) has been revoked. It seems to be no coincidence - as the point of time fits as well, but this particular version seems not be in use in my connection servers.

However, even with full logging activated, I cannot get the information which (intermediary) certificate cannot be validated and why. I would have expected to see something like "checking OCSP" or "checking CRL" but I cannot find it in the logs. Though, I noticed that one of the intermediary certificates was missing the OCSP URL (though the "Authority Information Access" field existed). Of course I updated the certificate with a version containing the OCSP URL but it did not change anything.

Additionally I manually checked all the certificates in the chain with openssl (for OCSP) and the CRL lists as well, but everything seems to be OK (all URLs are reachable and no used certificate has been revoked). In fact, as well, I do not interpret the error as "that the connection server does consider a certificate as invalid because it has been revoked", but "that it cannot check if it has been revoked". The servers do not need a proxy and there is none configured, as well (I checked proxy settings system context, too).

For the moment the problem is not critical, as the "red" status of the connection servers does not have an effect on our customers and as well I could turn off the certificate revocation checking (or switch it to only check the server certificate (2)). But of course I would like to really fix the problem.

Is there anyone who can give me a hint about what to check, e.g. how to find out which certificate cannot be checked and why? Has anyone had the same or a similar problem? VMware support works on the problem as well, but they seem to do not know the problem, neither.

I appreciate any thoughts and replies! Thanks!

Best regards,

Fabian

[marcbnz]

Hi,

I thought I'd jump on this thread to keep it alive as I too have a 6.2 environment that just 'went red' with the same CRL issue, yet each of the connection servers is able to download the .crl file they need.

The support KB method of disabling the CRL checking just doesnt sit right with me - I want to know what & why it happens - the KB also doesn't mention anything about Horizon 6.2.

i'm about to put a job in with vmware on this and i'll update anything i find.

[cH1LL1]

I'd be interested to see what you come up with... -  we disabled CRL

[Alethay]

Hi,

have you any feedback about that issue? I'm in the same situation and disable CRL checking doesn't fix it.

[Fab4UB]

Hi Alethay,

Depending on the problem, CRL checking should fix it - though it only is a workaround and not a real fix. However I really fixed it, see below.

Best regards,

Fabian

[Fab4UB · Accepted solution]

Dear community,

Meanwhile I was able to fix the error described in the beginning of this thread. Skip to the end to see what probably might help you...

1. At first I installed an additional standalone VMware View Connection Server in order to check the following about the certificates:
   1. The VMware support always told me to renew my certificates because they "weren't vaild" etc. - though in fact they were (as external URL calls and manual testing/checking proved).
   2. Therefore I created new certificates for the additional Connection Server and configured it to include the same vCenter as my production environment - only difference was that I did not inlcude the composer as that is running on the vCenter server itself.
   3. The result was that the server was "all green"  including as well the vCenter Server certificate which would be "untrusted" by the production environment  - strange, huh?
2. After that I reset the additional server to a snap shot where View Connection server still was not installed (before that I uninstalled the connection server it for the case there is some information within vCenter about it) and reinstalled it as a replica server of the production environment. Somehow I expected that, but again strange enough, the vCenter Server (and composer) now again was considered to be "invalid", though the certificate of the connection server itself still was considered as valid and green. For test purposes I therefore set certifice revocation checking to "2" (only check server certificate)  - but only on the "old" production servers" and "magically" everything was considered to be valid. So, as I see it, there seems to be some kind of information stored on the "old" connection servers which makes them consider the certificates to be invalid and that information is replicated to the third server unless I lower certificate revocation checking on these to servers. An altervative explanation might be that VMware View does not accept certificates with aliases which do not include the "real" server name - that in fact is/was the case for the old connection servers' certificates. The new connection server certificate included both the real name and the alias. I would understand if that is the case but then I would expect it to be documented somewhere (I did not find this information) and additionally would not understand why it worked without problems for several years before.
3. After finding that out, I created new certificates for the "old" connection servers including the aliases and the real names and replaced the certificate on one of the servers (and restarted the connection server) - with only some success. Once I set revocation checking to "4" again on that server, the connection server certificate still was considered to be valid, but not the vCenter and Composer Certificate.
4. Now I uninstalled this old connection server (removed it from view) and completely reinstalled it (including an OS upgrade from 2008 R2 to 2012 R2) and after I reintegrated it into the environment everything stayed green - as long as I did not reactivate revocation checking on the second "old" connection server. Therefore I did the same with this (completely reinstalled it and reintegrated it) and now everything is green with revocation checking enabled on all the connection server replicas.
5. As a next step I will uninstall the additional replica because I only created that for troubleshooting purposes.

So what probably will help in similar cases:

• Reinstall the connection servers one by one, including:
  • Uninstallation of html access (if used), uninstallation of view connection server, uninstallation of "VMware" AD LDS Instance.
  • Removal of Connection Server from replication group: run "vdmadmin.exe -S -r -s uninstalled_server_name" on one of the remaining connection servers.
  • Reinstall/upgrade OS (might be unnecessary but I did not test that)
  • Reininstall, reintegrate the connection server replica. If you used certificates which included only the server alias I would recommend to create new ones including the server name as well, but this might be unnecessary as well. If you want to keep certificates which only inlcude the alias it will be necessary to install that certificate after the first replication of the servers (see below).

My question to the VMware technicians/developers: Is it supported to use certificates which only include the server aliases. If not why did it work before and where is it documented? Where is certificate information cached so that simply replacing the certificate only was some and not a complete success (see above). For your information - when I initally paired the replicas I had to install the CA certificate (including only the alias) after the first replication - now, with the certificates (including server name and alias) I could install the certificate before replicating (=installing the connection server).