Our Saas Workspace ONE console (CN420) was updated to version 126.96.36.199 night. This morning we noticed that products assigned to our Android devices are not making it past a Queued status. We verified the issue by testing on 3 devices, with 3 products (that all worked prior to today), and on 2 different networks (1 wifi and 1 cellular). I've already opened a ticket with VMWare support this morning, but I'm just wondering if anyone else is seeing the same thing as I haven't heard back from them yet.
Example from one device where products have been queued for 7+ hours:
Almost identical situation here. We updated to the same build (188.8.131.52 (2111)) on Tuesday morning and were advised by our internal customer that one of the products assigned to our ruggedized Android devices (an internal app) was no longer being pushed. I checked the product dashboard and found the same thing - the app was stuck at "Queued" status. Oddly enough, the other products (Launcher, restructions, etc) seemed to be working.
I opened a ticket with VMWare technical support as well, and was advised that this is a known issue to them internally (but I do not believe it has been published externally yet) which seems to be related to use of a pull relay server (which we are using) for distribution of products. The internal document was RUGG-10555 in this instance. If you are not using ruggedized devices it may not specifically relate to what you are seeing, but it sounds like the same issue.
Thank you for the reply. I was just actually on the phone with Airwatch support discussing this (finally) and was told the same thing and given the same internal RUGG #. I provided them some files from our relay server as well so that they can confirm that what we're seeing is what they have documented.
Can you tell me what files they asked for? I am still in the initial phase of contact with support via email (gave them my CN number, etc) and have not yet spoken directly with anyone. I will try to pull the files in anticipation of the request.
They had asked for the log files from the Relay Server. I believe the files we grabbed were at C:/Airwatch/Pull Service Logs and were named PullServiceLog#.txt. Prior to pulling the logs they wanted some Products queued or force reprocessed and the time of doing that noted (so that they could find it easier in the logs).
I just heard further information back from Airwatch support regarding the issue. I've attached an image showing the part of the log they found that matched the known issue (RUGG-10555). They also mentioned that "R&D is actively engaged on this and working towards a resolution. However, there is a hot patch available as a fix. If you wish to upgrade your console to the fix patch, kindly let me know and I will engage our SaaS Ops team for the same."
Were you able to get this resolved? So far no one has actually admitted to a hotfix/patch being available for this issue. I did provide the log files from my relay server. The response I received this morning was that they believe if we disable the relay server and they enable CDN that will work, at least until the console is fixed at which point we could go back to the relay server if we choose.
The last I heard from VMWare support they had provided me some dates/times that would work for us where they could install the patch on our Saas environment. I had responded back yesterday and have not heard anything further. However, we noticed this morning that Products are now functioning again - so we're assuming the patch must have been implemented overnight and support just hasn't notified us yet.
I finally heard back from support earlier today. Apparently there is a fix, although it's not so much a hotfix as a full console update to a new version which includes the fix for this specific issue. We are scheduled to receive it tomorrow morning (02/02) during the early morning hours, so I will be testing first thing in the morning to see if the issue is resolves. I'll post here once I determine if the issue has been resolved by the update or not.
VMWare pushed a new build of the console ( 184.108.40.206 ) to us this morning which has resolved this issue. The devices which were previously showing "Queued" processed through automatically as soon as the system came back up, and subsequent testing confirmed it is working normally again. This is not a "hotfix" as they originally described it.. it's another full build of the console from what I can tell, with the incremental increase in the version number.