antherITguy
Enthusiast
Enthusiast

ENSv2 Notification Cloud Error

Has anyone seen the following error when setting up on-prem ENSv2? I'm not sure if this is a response from the CNS server out at VMware or if the unauthorized response is coming from one of my servers. This event is generating from the ReSubscriptionMechanism.log.


Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification
Call to Notification
Cloud failed for user :
2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g
Status : ProtocolError :: System.Net.WebException:
The remote server returned an error: (401) Unauthorized.
Labels (1)
0 Kudos
17 Replies
AndreNguyenAnd1
Contributor
Contributor

The log you are getting is from the ens v2 server. Hence the remote server in the log is not ENS server. It is either the exchnage server or the cns denied the call from ENS. Is it possible to provide the bigger logging for this? That way we can see at which path the call get denied. Then we can chase the cuase down
0 Kudos
antherITguy
Enthusiast
Enthusiast

Here are more logs:


2019/01/31 20:07:09.601 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification SendNotification Response : Unauthorized for user: 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g from Notification Cloud


2019/01/31 20:07:09.601 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification User Id:[Ready to send re-subscribe notification to user [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:09.601 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.CreateWebRequest CNS Url : [https://cns.awmdm.com/nws/notify/apns]


2019/01/31 20:07:09.601 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.CertificateHelper.ComputeCmsSignature Signing URL [/nws/notify/apns] with Cert [CN=AW Cloud Notification - Global]


2019/01/31 20:07:09.617 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.ComposePushNotification User Id:[Payload ready for User [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:10.039 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification Call to Notification Cloud failed for user : 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g Status : ProtocolError :: System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification._ReadResponseAsync(IAsyncResult asyncResult) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.ContextAwareResult.CaptureOrComplete(ExecutionContext& cachedContext, Boolean returnContext) at System.Net.ContextAwareResult.FinishPostingAsyncOp() at System.Net.HttpWebRequest.BeginGetResponse(AsyncCallback callback, Object state) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification(UserInfo user, NotificationServiceType notificationServiceType)


2019/01/31 20:07:10.039 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification SendNotification Response : Unauthorized for user: 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g from Notification Cloud


2019/01/31 20:07:10.039 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification User Id:[Ready to send re-subscribe notification to user [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:10.039 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.CreateWebRequest CNS Url : [https://cns.awmdm.com/nws/notify/apns]


2019/01/31 20:07:10.055 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.CertificateHelper.ComputeCmsSignature Signing URL [/nws/notify/apns] with Cert [CN=AW Cloud Notification - Global]


2019/01/31 20:07:10.055 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.ComposePushNotification User Id:[Payload ready for User [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:10.523 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification Call to Notification Cloud failed for user : 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g Status : ProtocolError :: System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification._ReadResponseAsync(IAsyncResult asyncResult) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.ContextAwareResult.CaptureOrComplete(ExecutionContext& cachedContext, Boolean returnContext) at System.Net.ContextAwareResult.FinishPostingAsyncOp() at System.Net.HttpWebRequest.BeginGetResponse(AsyncCallback callback, Object state) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification(UserInfo user, NotificationServiceType notificationServiceType)


2019/01/31 20:07:10.523 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification SendNotification Response : Unauthorized for user: 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g from Notification Cloud


2019/01/31 20:07:10.539 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification User Id:[Ready to send re-subscribe notification to user [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:10.539 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.CreateWebRequest CNS Url : [https://cns.awmdm.com/nws/notify/apns]


2019/01/31 20:07:10.539 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.CertificateHelper.ComputeCmsSignature Signing URL [/nws/notify/apns] with Cert [CN=AW Cloud Notification - Global]


2019/01/31 20:07:10.539 DS1 (25) Debug ReSubscriptionMechanism.BusinessImpl.ProcessNotification.ComposePushNotification User Id:[Payload ready for User [{0}]] 2b79868f23e5790e9366acadb09dd75e25c30b44db064b509eb75f8f2e65799c


2019/01/31 20:07:10.867 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification Call to Notification Cloud failed for user : 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g Status : ProtocolError :: System.Net.WebException: The remote server returned an error: (401) Unauthorized. at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification._ReadResponseAsync(IAsyncResult asyncResult) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.ContextAwareResult.CaptureOrComplete(ExecutionContext& cachedContext, Boolean returnContext) at System.Net.ContextAwareResult.FinishPostingAsyncOp() at System.Net.HttpWebRequest.BeginGetResponse(AsyncCallback callback, Object state) at ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification(UserInfo user, NotificationServiceType notificationServiceType)


2019/01/31 20:07:10.867 DS1 (25) Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification SendNotification Response : Unauthorized for user: 2b72368f23w5740e9366acadb09dd55e25c30b44dg064b5097b75f8f2e62299g from Notification Cloud


2019/01/31 20:07:10.867 DS1 (25) Info ReSubscriptionMechanism.ReSubscriptionMechanismService.ReSubscribeTrigger Scheduling Re-subscription Mechanism at [1/31/2019 8:07:10 PM]

0 Kudos
AndreNguyenAnd2
Contributor
Contributor

Yep, this is the last part on the architecture of the ENSv2 where the ENS server would send the notification of this user to CNS. The response from the CNS is ' denied, I don't know who you are' . ENS then keep trying periodically. That is why it is saying Resubscribe mechanism and will keep getting denied.

I recommend to contact Vmware Support to check why the CNS reject your ENSv2 server.
0 Kudos
antherITguy
Enthusiast
Enthusiast

I have a case open. Was told to purchase Professional Services.

0 Kudos
AndreNguyenAnd2
Contributor
Contributor

Ah i see! sorry to hear that.

Just reply to them that the install and planning is complete so we don't need to set up Professional Service to get help. This is clearly the CNS server rejected my ENS server. We don't have visibility into the CNS server that hosted by Vmware. This fall under support with Vmware.
0 Kudos
AlthafMashood
Contributor
Contributor

We are alos getting the same error.
@Derek, are you using Basic authentication for Boxer ?
0 Kudos
antherITguy
Enthusiast
Enthusiast

Certificate based auth
0 Kudos
AlthafMashood
Contributor
Contributor

We are still getting this error from time to time on resubscription logs. But we got ENS working for CBA Boxer after configuring ENS compliance on SEG configuration as mentioned in the KB below.
PS: We had to restart SEG service after modifying the ENS proxy value.

https://docs.vmware.com/en/VMware-Workspace-ONE-UEM/9.7/vmware-airwatch-guides-97/GUID-AW97-SEGv2_EN...
0 Kudos
Mario_Giese
Contributor
Contributor

Hi,

the steps discribed by Althaf in the KB article are only neccessary of your EWS Server is not reachable from the devices and you are going through the SEG with the EWS traffic.
The error discribed here is like Andre mentioned the problem with the AirWatch hosted CNS Server.
We had this on several ENS servers too and just had to open a support ticket and attach the following file: Console > All Settings > System> Advanced > Secure Channel Certificate > Download CNS Secure Channgel Certificate Installer.  This file has to be uploaded on CNS server via Support team and after that the ENS server will get an ' OK'  on sending the push notifications.
0 Kudos
antherITguy
Enthusiast
Enthusiast

Turned out that the CNS certificate I provided to VMware was uploaded to the wrong CNS endpoint.  Once this was corrected the 401 errors stopped.
0 Kudos
nemanjailic
Contributor
Contributor

Hello,
i have problem with ENS for 6 monts. I'm not able to configure it to work. Support told me to purchase Professional Services. I install ENS 1.3.0.4, and configure it all by the book. I find erron in ReSubscriptionMechanism:

Error ReSubscriptionMechanism.BusinessImpl.ProcessNotification.SendNotification Call to Notification Cloud failed for user : c470ecec5c03b1fdce312d37b23c24525b4b4c37e69a96afebd447b9811241ff Status : ProtocolError :: System.Net.WebException: The remote server returned an error: (400) Bad Request.

Does anybody know what this mean?
I send CNS cert 2 times to support tu upload it.
Thanks,

Nemanja
0 Kudos
ArianZuta
Contributor
Contributor

Hi Nemanja
I am trying to get the ENSv2 working but got the same issue as you.

I posted the following in another article fyi:

If you want to use your SEG as a EWS proxy you need to change the application.properties file on the SEG and set the enable.boxer.ens.ews.proxy=true. However i still have on the SEG issues saying that:

- Request Device not present in the request header
- Error serving request on path /EWS/Exchange.asmx

If I try to access the SEG Url https://seg_url/EWS/Exchange.asmx I get a white screen. So there is a connection made but i don't get the prompt where username/password is asked. How does that look like on your side? Is anybode using SEG as a EWS proxy to the Exchange?

Additional on the ENS (on-prem) I have the following issues:
ReSubscriptionMechanism.log:
- CNS Url : https://cns.awmdm.com/nws/notify/apns
- Call to Notification Cloud failed for user : GUID Status: ProtocolError :: The remote server returned an error: (400) Bad request.

In the AutoDiscoveryChecker.log file I see that awtrustdiscovery.awmdm.com is being accessed. However VMware states nowhere that awtrustdiscovery.awmdm.com is needed. Do you allow the connection to the awtrustdiscovery.awmdm.com? What role does the awtrustdiscovery.amwmd.com play?

Sorry for the spam.
Best Arian
0 Kudos
Timovartiainen
Contributor
Contributor

I'm having a problem getting Clound based ENSv2 to work.
• After making the ENS accessible, we also enabled Basic authentication for the EWS endpoint on the IIS of Exchange. (This caused problems on our Outlook clients)
• Now the ENS file is getting generated and we could still see 401 coming from the Exchange.
• In the IIS log file as well, I could see 401 coming for the /EWS endpoint.
These bullet points were gathered by VMware support and basically stated that some Exchange setting is causing the 401 error that prevents ENS from working.  They enabled basic authentication on a remote support session and we had a lot of issues on outlook clients after that so we had to switch it back to default setting.
Can you tell what are the required settings on the Exchange 2016 (on premises) to make ENSv2 cloud work with our Boxer clients?
0 Kudos
ArianZuta
Contributor
Contributor

Hi Timo
Can you access https://mailserver/EWS/Exchange.asmx from outside?
I have setup ENSv2 cloud for several customers without any issues.

Best Arian
0 Kudos
Timovartiainen
Contributor
Contributor

If you mean with browser then no, I can't access the page from outside. We are not publishing Exchange (OWA) for external access either.  The installation was done by a consulting company that have setup ENSv2 before but somehow it doesn't work for us.
When you open Boxer client on a mobile device, a notification is displayed on the bottom of the screen stating that ' Could not update settings for the push notification service.'  and then there is a link ' More info'  where you can send or copy the logs.
I think the problem here is that Boxer is trying to use Basic authentication and EWS Virtual directory has the authentication set to Integrated Windows authentication. Can you tell me what the preferred security settings would be for Exchange 2016 (on prem) for the ENSv2 cloud version?
0 Kudos
ArianZuta
Contributor
Contributor

Hi Timo
The devices have to have a connection to /EWS. You either expose /EWS to the Internet or you use SEG as a proxy to /EWS (you can configure SEG to proxy /EWS).
However I would suggest to expose /EWS directly. You can still have /OWA unaccessible from outside.
What do you mean with ' preferred security settings'  in this scenario?
Best Arian
0 Kudos
Timovartiainen
Contributor
Contributor

We have the enable.boxer.ens.ews.proxy=true in the config file (SEG working as proxy).

What I mean by security settings is the virtual directory security settings required by ENSv2 to work. When using the defaults by Exchange 2016 (Intergrated Windows authentication), VMware support told us that the tenant end wasn't logging anything. When that was changed to Basic in the IIS they started to get data in the logs:
  {
    ' message' : ' 2019-10-22 10:02:20.5882|DEBUG|MailNotificationService.Controllers.EnsController.RegisterDeviceV2|d4cd335b-13bd-491f-ae06-4ce1c883ffbf|User Id:[67a0f8ba3926403339e7eeb0177bc0ce28db9b60b6af78bcc6a5fdca8f3810e8] or tRegister device request processed. HttpStatusCode:[Conflict] ResponseCode:[SubscribeAgain]' ,
    ' date' : 1571738540588,
    ' dateString' : ' 2019-10-22 05:02:20 am CDT'
  },
  {
    ' message' : ' 2019-10-22 10:02:20.5882|WARN|MailNotificationService.BusinessImpl.SubscriptionBusiness.SubscribeV2Async|d4cd335b-13bd-491f-ae06-4ce1c883ffbf|User Id:[67a0f8ba3926403339e7eeb0177bc0ce28db9b60b6af78bcc6a5fdca8f3810e8] or tService request exception occured  for userId [67a0f8ba3926403339e7eeb0177bc0ce28db9b60b6af78bcc6a5fdca8f3810e8], Inner exception message [The remote server returned an error: (401) Unauthorized.] Going for a retry,' ,
    ' date' : 1571738540588,
    ' dateString' : ' 2019-10-22 05:02:20 am CDT'
  },.

However changing the security settings in IIS caused problems with shared calendars so we had to revert back to Windows authentication.
So VMware support told us to sort out Exchange security settings but I have no idea what the ' correct'  security settings would be to get this to work.
0 Kudos