WS1 UEM 2011 on-prem is not able to connect to win10 baselines

[msweisberg18]

I currently have an 2011 on-prem deployment that is working quite well except for win10 baselines.

When going to devices>profiles&resources>baselines, I get this error "unable to connect to baseline service https://www.vmwarebaselines.com. please check your connectivity and logs for more information".

If I look at the event viewer on the DS console server (which is where the api is at), this is what I see:

Identity:auth:JWT:Administrator
Module:AW.WebApi.Common.AwExceptionLogger.LogKnownException
Message: AirWatchVersion: 20.11.0.7; AuthenticationType: JWT; RequestUri: https://mdm.XXXXXX.com/api/mdm/baselines/osversions; RequestMethod: GET; ErrorCode: 5186; HttpStatusCode: 5186; Message: Unable to fetch signing certificate for microservice; ClientIP: 10.0.1.229; ResponseTime: 1000.012ms; RequestHeaders: Cache-Control: no-cache, Connection: keep-alive, Pragma: no-cache, Accept: application/json, text/plain, */*, Accept-Encoding: gzip, deflate, br, Accept-Language: en-US, en; q=0.9, Host: mdm.XXXXXXX.com, Referer: https://mdmadmin.XXXXXXXXX.com/, User-Agent: Mozilla/5.0, (Windows NT 10.0; Win64; x64), AppleWebKit/537.36, (KHTML, like Gecko), Chrome/87.0.4280.141, Safari/537.36, sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87", sec-ch-ua-mobile: ?0, X-Requested-With: XMLHttpRequest, aw-tenant-code: mPAJQ**********************************UTxk=, Origin: https://mdmadmin.uemarchitect.com, Sec-Fetch-Site: same-site, Sec-Fetch-Mode: cors, Sec-Fetch-Dest: empty, ; ResponseHeaders: , ThrottlingLimits: ; Username: Administrator*** EXCEPTION ***
AW.WebApi.Common.AwException: Unable to fetch signing certificate for microservice   at AW.Mdm.Api.Controllers.OSVersionsV1Controller.<GetActiveOSVersionsImplAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at AW.Mdm.Api.Controllers.OSVersionsV1Controller.<GetActiveOSVersions>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__1`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Web.Http.Filters.ActionFilterAttribute.<CallOnActionExecutedAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.ActionFilterAttribute.<ExecuteActionFilterAsyncCore>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Filters.AuthorizationFilterAttribute.<ExecuteAuthorizationFilterAsyncCore>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.AuthenticationFilterResult.<ExecuteAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__6.MoveNext()MS_LoggedBy:System.Collections.Generic.List`1[System.Object];AW_Logging_Exception_Logged:True;AW_Logging_Exception_Data_Count:2;AW_Logging_Exception_Log_Message:AirWatchVersion: 20.11.0.7; AuthenticationType: JWT; RequestUri: https://mdm.XXXXXXX.com/api/mdm/baselines/osversions; RequestMethod: GET; ErrorCode: 5186; HttpStatusCode: 5186; Message: Unable to fetch signing certificate for microservice; ClientIP: 10.0.1.229; ResponseTime: 1000.012ms; RequestHeaders: Cache-Control: no-cache, Connection: keep-alive, Pragma: no-cache, Accept: application/json, text/plain, */*, Accept-Encoding: gzip, deflate, br, Accept-Language: en-US, en; q=0.9, Host: mdm.XXXXXXXXX.com, Referer: https://mdmadmin.XXXXXXXX.com/, User-Agent: Mozilla/5.0, (Windows NT 10.0; Win64; x64), AppleWebKit/537.36, (KHTML, like Gecko), Chrome/87.0.4280.141, Safari/537.36, sec-ch-ua: "Google Chrome";v="87", " Not;A Brand";v="99", "Chromium";v="87", sec-ch-ua-mobile: ?0, X-Requested-With: XMLHttpRequest, aw-tenant-code: mPAJQ**********************************UTxk=, Origin: https://mdmadmin.XXXXXXX.com, Sec-Fetch-Site: same-site, Sec-Fetch-Mode: cors, Sec-Fetch-Dest: empty, ; ResponseHeaders: , ThrottlingLimits: ; Username: Administrator;WanderingWiFi.AirWatch.BusinessImpl.Cryptography.SigningServiceIdentityCertificateException: Identity certificate not found or does not have private key   at WanderingWiFi.AirWatch.BusinessImpl.Cryptography.SigningServiceIdentityCertificateRepository.GetIdentityCertificate()
   at WanderingWiFi.AirWatch.BusinessImpl.Cryptography.SigningServiceCertificateBusiness.<RequestSignedCertificateAsync>d__33.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WanderingWiFi.AirWatch.BusinessImpl.Cryptography.SigningServiceCertificateBusiness.<GetCertificateAsync>d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WanderingWiFi.AirWatch.BusinessImpl.Cryptography.SigningServiceCertificateBusiness.<GetMicroserviceIdentityCertificateAsync>d__15.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WanderingWiFi.AirWatch.BusinessImpl.Device.Baseline.BaselineApiClientAuthenticator.<CreateAsync>d__10.MoveNext()AW_Logging_Exception_Logged:True;AW_Logging_Exception_Data_Count:1;AW_Logging_Exception_Log_Message:;

Now, the ETL airwatch microservice certificate is installed and present on the server, so not sure what the problem is.

[Noordan]

Hi,

I have seen exact the same issue. what is the ETL airwatch microservice certificate?

[yohancarrier]

Hello, did you find anything ? 

Have the same exact issue on our on-prem setup.

[NoWayJosue]

Ensure that both the console and device services servers can reach out to signing.awmdm.com over HTTPS on port 443. You can see all of the requirements at https://ports.esp.vmware.com/home/Workspace-ONE-UEM then search "baseline". 

[Noordan]

in my case we did not have all AirWatch certificates in the database. 
When we run the three queries below, we did not get a response from all of them. If I remember correctly, we were missing a response from the first one.

Select * From dbo.SystemCodeOverride
Where SystemCodeID = 5172
 
Select * From dbo.Systemcodecategory
Where name like '%identity%'
 
Select * From dbo.SystemCodeGroup
Where SystemcodeGroupID = 407

And the solution that the support gave to us were to re the certinstaller.exe -t {token} tool manually from one of the application servers. where the {token} where replaced with a new token from my.workspaceone.com. And after that we have response on all three queries.

Make sure to take backup of database and snapshots of application servers before.