drew1994yj
Enthusiast
Enthusiast

vCenter SSO Service is stuck on old vCenter SSL Cert

Hi All,

vCenter 5.1 (Windows VM)

vCenter Server, SSO, and the Inventory Service are all installed in separate VM's.

I have been fighting with this for a while now.  Our vCenter Server SSL certificate expired.  Using KB articles, I was able to replace the vCenter Server certificate and bring vCenter fully online.  Everything works great when using the Windows vSphere Client.

However, when using the web client (by way of vCenter SSO), Zero vCenter Servers show up in my inventory.  This is when logging in with an account that has rights in vCenter, not the built-in admin@system-domain account.  I have followed every KB article I could find (including repointing/reregistering, and using the Certificate Automation Tool).

In the SSO log (LookupServer.log), I see the errors copied below.  Note (the certificate expiration date listed in the error log is EXACTLY when my previous vCenter Server cert expired).  So it looks like the old cert is stuck in a JKS somewhere.

Thank you!

Andrew

[2013-05-21 14:10:20,381 DEBUG opID=3432b050-dc77-49ca-acb7-6bfb984107e9 pool-4-thread-1  com.vmware.vim.vmomi.server.impl.InvocationTask] Invoking com.vmware.vim.binding.lookup.LookupService.find

[2013-05-21 14:10:20,381 DEBUG opID=3432b050-dc77-49ca-acb7-6bfb984107e9 pool-4-thread-1  com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Executing  find services(com.vmware.vim.binding.lookup.SearchCriteria:

serviceType = urn:vc,

viSite = null,

endpointProtocol = null

inherited from com.vmware.vim.binding.lookup.SearchCriteria@4611970c)

[2013-05-21 14:10:20,381 DEBUG opID=3432b050-dc77-49ca-acb7-6bfb984107e9 pool-4-thread-1  com.vmware.vim.lookup.impl.DbStorage] Executing SELECT LS_SERVICE.ID, LS_SERVICE.OWNER_ID, LS_SERVICE.VERSION, LS_SERVICE.DESCRIPTION, LS_SERVICE.FRIENDLY_NAME, LS_SERVICE.SERVICE_TYPE, LS_SERVICE.PRODUCT_ID, LS_SERVICE_ENDPOINT.URI, LS_SERVICE_ENDPOINT.SSL_TRUST_ANCHOR, LS_SERVICE_ENDPOINT.PROTOCOL, LS_SERVICE_ENDPOINT.SERVICE_ID FROM LS_SERVICE LS_SERVICE LEFT JOIN LS_SERVICE_ENDPOINT LS_SERVICE_ENDPOINT ON LS_SERVICE.ID = LS_SERVICE_ENDPOINT.SERVICE_ID WHERE 1=1 AND LS_SERVICE.SERVICE_TYPE = ?

[2013-05-21 14:10:20,381 ERROR opID=3432b050-dc77-49ca-acb7-6bfb984107e9 pool-4-thread-1  com.vmware.vim.lookup.util.ValidateUtil] Invalid certificate

[2013-05-21 14:10:20,381 ERROR opID=3432b050-dc77-49ca-acb7-6bfb984107e9 pool-4-thread-1  com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Failed to find services(com.vmware.vim.binding.lookup.SearchCriteria:

serviceType = urn:vc,

viSite = null,

endpointProtocol = null

inherited from com.vmware.vim.binding.lookup.SearchCriteria@4611970c) because of Invalid certificate

java.lang.IllegalArgumentException: Invalid certificate

    at com.vmware.vim.lookup.util.ValidateUtil.logAndThrow(ValidateUtil.java:214)

    at com.vmware.vim.lookup.util.ValidateUtil.validateCertificate(ValidateUtil.java:201)

    at com.vmware.vim.lookup.ServiceEndpoint.<init>(ServiceEndpoint.java:52)

    at com.vmware.vim.lookup.impl.DbStorage.constructServices(DbStorage.java:538)

    at com.vmware.vim.lookup.impl.DbStorage.access$400(DbStorage.java:53)

    at com.vmware.vim.lookup.impl.DbStorage$4.action(DbStorage.java:231)

    at com.vmware.vim.lookup.impl.DbStorage$4.action(DbStorage.java:216)

    at com.vmware.vim.lookup.impl.DbStorage$SingleSqlExecutor.action(DbStorage.java:774)

    at com.vmware.vim.lookup.impl.DbStorage$SqlExecutor.execute(DbStorage.java:701)

    at com.vmware.vim.lookup.impl.DbStorage.find(DbStorage.java:216)

    at com.vmware.vim.lookup.impl.LookupServiceImpl.find(LookupServiceImpl.java:40)

    at com.vmware.vim.lookup.vlsi.LookupServiceImpl$4.call(LookupServiceImpl.java:171)

    at com.vmware.vim.lookup.vlsi.LookupServiceImpl$4.call(LookupServiceImpl.java:167)

    at com.vmware.vim.lookup.vlsi.util.VmodlEnhancer.invokeVmodlMethod(VmodlEnhancer.java:82)

    at com.vmware.vim.lookup.vlsi.LookupServiceImpl.find(LookupServiceImpl.java:167)

    at sun.reflect.GeneratedMethodAccessor245.invoke(Unknown Source)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

    at java.lang.reflect.Method.invoke(Unknown Source)

    at com.vmware.vim.vmomi.server.impl.InvocationTask.run(InvocationTask.java:76)

    at com.vmware.vim.vmomi.server.common.impl.RunnableWrapper$1.run(RunnableWrapper.java:48)

    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

    at java.lang.Thread.run(Unknown Source)

Caused by: java.security.cert.CertificateExpiredException: NotAfter: Wed May 08 11:26:08 EDT 2013

    at sun.security.x509.CertificateValidity.valid(Unknown Source)

    at sun.security.x509.X509CertImpl.checkValidity(Unknown Source)

    at sun.security.x509.X509CertImpl.checkValidity(Unknown Source)

    at com.vmware.vim.lookup.util.ValidateUtil.validateCertificate(ValidateUtil.java:199)

    ... 21 more

6 Replies
JakeP
Enthusiast
Enthusiast

I have a case open for this exact issue. Let me know if you find anything and I will do the same.

0 Kudos
drew1994yj
Enthusiast
Enthusiast

Will do, I opened a case this morning.


Drew

0 Kudos
JakeP
Enthusiast
Enthusiast

I just got off the phone with support and in my case they want me to do a full re-install of all components after deleting/renaming installation folders. They want to create a solid environment with self-signed certs and then update them all with the custom certs. I'll likely just build a replacement VM with fresh installs and then update the certs.

0 Kudos
drew1994yj
Enthusiast
Enthusiast

Support recommended I delete the Inventory Service DB and recreate it, then re-register vCenter to the SSO Service and Inventory Service.  I completed all of the above and still have the issue.  My LookupServer.log still shows errors about the expired certificate.

Andrew

0 Kudos
JakeP
Enthusiast
Enthusiast

I reubilt my vCenter server using the same hostname and then reinstalled all of the certs, everything is working correctly now.

drew1994yj
Enthusiast
Enthusiast

Thanks for the reply!  I edited the SSO SQL database and found two old records for vCenter based on the old SSL cert.  I deleted those entries, then re-registered vCenter with the Inventory Service.  Then I restarted all of the services, and it is working now!

Drew