When attempting to do a failover test with SRM 8.2.0.2, I'm getting the error "Guest operation authentication failed for operation Create Temporary Directory." on all 40ish VMs.
- We're aware of the time sync issue shown in the release notes, and have confirmed all components are in sync.
- We have a ticket open with GSS, but we're getting nowhere slow.
- VGAuthsvc is reporting the following:
[2020-06-11T14:56:10.399Z] [ message] [VGAuthService] requestType: 10(VALIDATE_SAML_BEARER_TOKEN REQ)
[2020-06-11T14:56:10.399Z] [ message] [VGAuthService] username ''
[2020-06-11T14:56:10.400Z] [ message] [VGAuthService] validate Only 'FALSE'
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecOpenSSLX509StoreVerify:file=x509vfy.c:line=341:obj=x509-store:subj=unknown:error=71:certificate verification failed:X509_verify_cert: subject=/CN=ssoserverSign,dc=vsphere,dc=local/C=US; issuer=/CN=CA, CN=<redacted>, dc=vsphere,dc=local/C=US; err=20; msg=unable to get local issuer certificate
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecOpenSSLX509StoreVerify:file=x509vfy.c:line=341:obj=x509-store:subj=unknown:error=71:certificate verification failed:X509_verify_cert: subject=/C=US/ST=California/L=San Jose/O=<redacted>/OU=Engineering-vCenterSSO-<redacted>/CN=<redacted>; issuer=/C=US/ST=California/L=San Jose/O=<redacted>/OU=Engineering-vCenterSSO-<redacted>/CN=<redacted>; err=20; msg=unable to get local issuer certificate
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecOpenSSLX509StoreVerify:file=x509vfy.c:line=380:obj=x509-store:subj=unknown:error=71:certificate verification failed:subject=/C=US/ST=California/L=San Jose/O<redacted>/OU=Engineering-vCenterSSO-<redacted>/CN=<redacted>; issuer=/C=US/ST=California/L=San Jose/O=<redacted>/OU=Engineering-vCenterSSO-<redacted>/CN=<redacted>; err=20; msg=unable to get local issuer certificate
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecKeysMngrGetKey:file=keys.c:line=1246:obj=unknown:subj=xmlSecKeysMngrFindKey:error=1:xmlsec library function failed:
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecDSigCtxProcessKeyInfoNode:file=xmldsig.c:line=790:obj=unknown:subj=unknown:error=45:key is not found:details=NULL
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecDSigCtxProcessSignatureNode:file=xmldsig.c:line=503:obj=unknown:subj=xmlSecDSigCtxProcessKeyInfoNode:error=1:xmlsec library function failed:
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] XML Error: func=xmlSecDSigCtxVerify:file=xmldsig.c:line=341:obj=unknown:subj=xmlSecDSigCtxSignatureProcessNode:error=1:xmlsec library function failed:
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] Signature verify failed
[2020-06-11T14:56:10.400Z] [ warning] [VGAuthService] Failed to verify Signature
[2020-06-11T14:56:10.400Z] [ message] [VGAuthService] Returning error message '<?xml version="1.0" encoding="UTF-8" ?><reply><sequenceNumber>1</sequenceNumber><errorCode>12</errorCode><errorMsg>validateSamlToken failed</errorMsg></reply>'
[2020-06-11T14:56:10.401Z] [ message] [VGAuthService] ServiceProtoDispatchRequest: processed reqType 10(VALIDATE_SAML_BEARER_TOKEN REQ), returning 0 on connection 11
-I've confirmed that the user mapping is being configured with a cert.
- SRM's logging reports this:
2020-06-11T15:28:15.585-04:00 verbose vmware-dr[01353] [SRM@6876 sub=Recovery.GuestOpsHelper ctxID=c3d037a7 opID=86355df4-0a03-4064-9703-b917103195c5-test:a090:c971:5537:587f:026b:d581] Exporting certificate for C=US,CN=ssoserverSign\,dc=vsphere\,dc=local
into '/tmp/vmware-srm225.pem'; Data:
--> '-----BEGIN CERTIFICATE-----
<redacted>
--> -----END CERTIFICATE-----
----
From what I can tell, the issue is this certificate being used to authenticate to the guest. However, I cannot find the source of this cert anywhere on the SRM appliance. For good measure, we reset all the certs using the vCenter VMCA script and reconnected everything, yet this cert keeps getting sent out.
I'd really appreciate if someone can give me a clue where this is and how I can replace it.
