I attempted to replace my machine cert on my VCSA server. After a few attempts I gave up and performed a full certificate reset using the `/usr/lib/vmware-vmca/bin/certificate-manager` tool.
Now I'm seeing the following errors in the UI when looking at any Health or vSAN information. Anyone know how to resolve this? I upgraded to 6.7.0.40000 and that didn't help.
in my /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log I see lots of:
Caused by: com.vmware.vsphere.client.vsandp.core.sessionmanager.common.NotAccessibleException: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint doesn't match
at com.vmware.vsphere.client.vsandp.core.sessionmanager.common.PbmClient.getConnection(PbmClient.java:70)
at com.vmware.vsphere.client.vsan.base.impl.PbmDataProvider.getProfileIds(PbmDataProvider.java:181)
at com.vmware.vsphere.client.vsan.base.impl.PbmDataProvider.getStoragePolicies(PbmDataProvider.java:131)
at com.vmware.vsphere.client.vsan.base.impl.PbmDataProvider.getObjectCompatibleStoragePolicies(PbmDataProvider.java:118)
... 119 common frames omitted
Caused by: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint doesn't match
at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:256)
at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:56)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:226)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:106)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:629)
...
Looks like there is a mismatch between machine ssl and endpoints certs. Please open a SR with GSS.
Looks like there is a mismatch between machine ssl and endpoints certs. Please open a SR with GSS.
Vijay2027 thanks for your quick response. Unfortunately putting in a SR isn't an option right now. Do you know how to check and/or manually fix the endpoint certs?
I have the same issue, do you have a solution for that issue already?
Thank you!
I wish I did. When did these issues start for you?
For me, they first started when I tried using a machine cert created by Let's Encrypt. There were problems with this certificate. I ended up resetting all of the certificates. Since doing this, I've been plagued by issues.
I have the issues since I changed the VCSA certificate authority as a sub cert to my AD CA...
But I didn't tried to reset this until now.
Is it for you also working in the flash web client?
Wow, surprisingly the flash client works!
I do see some errors in the /var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log like:
[2019-10-28T11:05:28.557-07:00] [WARN ] http-nio-9090-exec-10 c.vmware.vim.vmomi.client.http.impl.AllowKnownThumbprintVerifier Mismatched thumbprint 20:00:72:BA:3E:85:D4:93:A2:78:A4:83:62:2C:62:6C:4E:46:64:FF, rejecting connection
but it doesn't seem to affect the operation of the client
I see this error as well with the flash client, but I'm not sure if this is because of the other issue or something the flash client is doing, because I'm getting the same errors also with pages that are working in the HTML 5 client.
I'm leaving the office for today but tomorrow I will try to reset the certificate back to an VCSA self singed one, maybe this will change something.
Did you try this?
Remember to make a backup (at least snasphot of your vCenter) before you do anything to be able to go back
Yes. see the second sentence in the original message
Before we check endpoints can you move the contents of the below folders to a backup location (Ex: /storage/core) and restart vsphere-client and vsphere-ui services.
usr/lib/vmware-vsphere-client/server/work
/usr/lib/vmware-virgo/server/pickup
thanks @Vijay2027, I moved out /usr/lib/vmware-vsphere-client/server/work but I didn't have a folder called /usr/lib/vmware-virgo/server/pickup in my install. no change after restart of both services.
root@vcenter [ /usr/lib/vmware-virgo/server ]# ls -la
total 116
drwxr-xr-x 10 root root 4096 Oct 28 10:02 .
drwxr-xr-x 3 root root 4096 Oct 10 05:57 ..
drwxr-xr-x 2 root root 4096 Oct 28 10:02 about_files
-rwxr-xr-x 1 root root 5588 May 30 11:30 About.html
-rwxr-xr-x 1 root root 3140 May 30 11:30 AboutKernel.html
-rwxr-xr-x 1 root root 4381 May 30 11:30 AboutNano.html
drwxr-xr-x 2 root root 4096 Oct 28 10:02 admin
-rwxr-xr-x 1 root root 14547 May 30 11:30 artifacts.xml
drwxr-xr-x 2 root root 4096 Oct 28 10:02 bin
drwxr-xr-x 4 root root 4096 Oct 28 10:02 configuration
-rwxr-xr-x 1 root root 12567 May 30 11:30 epl-v10.html
drwxr-xr-x 4 root root 4096 Oct 28 10:02 lib
-rwxr-xr-x 1 root root 8783 May 30 11:30 notice.html
drwxr-xr-x 4 root root 4096 Oct 10 05:57 p2
drwxr-xr-x 2 root root 12288 Oct 28 10:02 plugins
drwxr-xr-x 3 root root 4096 Oct 10 05:57 repository
-rwxr-xr-x 1 root root 3616 May 30 11:31 vmware-changes.txt
I've sent you few commands in DM. Pls check.
Thanks again for your help Vijay2027 - you were right, it was definitely a complex and not self-fixable issue. I had a number of services that were running with the wrong certificate. GSS provided me with a script that fixed them all.
Hi Vijay2027, could you please also send me the commands? I have exact the same problem.
Thank you in advance!
Hi,
exactly same problem here, can i have the commands too please ?
This is a complex process. GSS has automated the process to fix certificate mismatch. Please open a SR.
Yow will have to follow the process as per VMware Knowledge Base to fix the mismatch.
I finally found out that it is a HTML5 problem. I do not have this issue with the Flex client !
The issue will be at service registration of HTML client. You will still have to get this corrected.
reboot of vcenter and the problem disappeared...