12 Replies Latest reply on Aug 9, 2019 6:07 AM by Ray_handels

    App Readiness going crazy after upgrade to 2.16+

    Sabian0309 Novice

      I have an open support case, but i thought i'd ask if anyone else has seen this issue.

       

      Over a weekend we updated our appvolume server from 2.14 to 2.16.  The following monday our VDI cluster was screaming.  The cause was the app readiness service crawling the appvolume's non-stop using 25-40% cpu constantly. 

       

      I tested it with old appvols as well as newly created.  Even tried the 2.17 agent with no change in behavior.  Anyone else notice their cpu usage spike after going with 2.16+?

        • 1. Re: App Readiness going crazy after upgrade to 2.16+
          VDINinja311 Enthusiast

          Sabian0309

           

          I thought we were alone in this issue! We have had an SR open with AppVolumes support since 5/10/2019 for this same issue. Can confirm its happening with both the 2.16 or 2.17 agents. We upgraded the servers and agents from 2.14.2 as well to 2.16 and this is when we noticed it started. We have since been using the 2.15 agent as a workaround.

           

          It seems random because just a few weeks ago we were able to do an upgrade from the 2.15 to 2.16 agent on some of our master images and on our UAT or GPU desktop pools we were not seeing the issue so we were going to deploy it to our production pools and immediately after recomposing we saw it go nuts again, so we recomposed back.

           

          Windows 10 1803 Linked clones here. You also on linked clones?

          • 2. Re: App Readiness going crazy after upgrade to 2.16+
            Sabian0309 Novice

            We are running 1803 instant clones.  We tried using the 2.17 agent and it didn't resolve the issue.  I guess we could of rolled back to 2.15 however we created an executable of a ps1 script to add windows.immersivecontrolpanel appx package at login through UEM which resolved the settings not launching. 

             

            Sorry you're having the issue as well, but I'm glad I'm not the only one .

            • 3. Re: App Readiness going crazy after upgrade to 2.16+
              VDINinja311 Enthusiast

              we created an executable of a ps1 script to add windows.immersivecontrolpanel appx package at login through UEM which resolved the settings not launching.

               

              What do you mean by the settings not launching?

              • 4. Re: App Readiness going crazy after upgrade to 2.16+
                Sabian0309 Novice

                To resolve the issue I disabled the app readiness service.  Which did the trick, but then if someone tried to click the cog off the start menu nothing would happen.  Somehow having that service disabled broke the settings metro app.  We found that if you right-clicked and selected personalize it would work for the session (probably a bit different as you're using linked clones).  However that wasn't a great work around for a lot of our users, so we implemented redeploying the appx package for it.

                • 5. Re: App Readiness going crazy after upgrade to 2.16+
                  hermanc01 Enthusiast

                  We're having the same issue in our environment as well with 2.16 & 2.17.

                   

                  In the meantime, I wrote a PowerShell script to stop the app readiness service 15 minutes after the logon process finishes.  This way the service can do it's thing without breaking anything for the users but then stops the CPU spikes.

                  • 6. Re: App Readiness going crazy after upgrade to 2.16+
                    VDINinja311 Enthusiast

                    hermanc01

                     

                    Did you just set up a Scheduled Task on the Master Image to run the script 15 minutes after any user logon?

                    • 7. Re: App Readiness going crazy after upgrade to 2.16+
                      hermanc01 Enthusiast

                      Yep that's exactly what I did.  I had it run as the SYSTEM account and delayed it to run after 15 minutes.

                       

                      Here's the following script I used.  I added a check to exclude it from stopping the services if I logged into the provisioning VM as our provisioning account too.

                       

                      $services = "AppReadiness", "AppXSvc"
                      
                      if ( (Get-WmiObject -Class Win32_Process -Filter 'Name="explorer.exe"').GetOwner().User -ne "OurProvisioningAccount" )
                      {
                          foreach ( $service in $services )
                          {
                              if ( Get-Service -Name $service -ErrorAction SilentlyContinue )
                              {
                                  Stop-Service $service -Force
                              }
                          }
                      }
                      
                      • 8. Re: App Readiness going crazy after upgrade to 2.16+
                        Ray_handels Master
                        Community WarriorsvExpert

                        I believe that Windows 10 uses that service to check and see if there are updates on Modern Apps available in the store every time a user logs in, not 100% sure though.

                        We have decided to remove the Windows Store modern app from our GI. If you don't need that it might be worth a shot. If you have a lot of modern apps users need in your environment it might not be feasible to go that route.

                        • 9. Re: App Readiness going crazy after upgrade to 2.16+
                          Sabian0309 Novice

                          Alrighty, i got some traction with my SR on this issue.  While it isn't perfect, there is a bit of a work around that vmware has provided.

                           

                          What they suggested:

                           

                          Edit the Snapvol.cfg by creating a blank app stack and attach the app stack to user or machines to verify
                          - Or Try updating an existing App Stack with the below config

                          exclude_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx

                          include_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx\AppxAllUserStore\Applications

                          exclude_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Appx

                          exclude_registry=\REGISTRY\MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppReadiness

                          This does appear to work.  You don't have to do it to every appstack, from my understanding as long as this change has been made to one of the app stacks being applied it will be applied to all.  In my testing this has resolved the issue, hopefully they can update their template.

                          • 10. Re: App Readiness going crazy after upgrade to 2.16+
                            sjesse Master
                            vExpert

                            Doesn't it depend on the order of the appstacks being attached, the last one winning, I remember having to do a similar fix for office awhile back that similar and we had to do it to all of them. That was 2.10 I think so it may have changed, I stay away from snapvol.cfg changes in general unless support suggests it so I'm not sure.

                            • 11. Re: App Readiness going crazy after upgrade to 2.16+
                              Sabian0309 Novice

                              When i asked (as i really didn't want to go through and modify a ton of apps), i was told it was cumulative.  I'm still asking if this will be patched... as its a hokey work around, but slightly less so than killing off services .

                              • 12. Re: App Readiness going crazy after upgrade to 2.16+
                                Ray_handels Master
                                Community WarriorsvExpert

                                When i asked (as i really didn't want to go through and modify a ton of apps), i was told it was cumulative.  I'm still asking if this will be patched... as its a hokey work around, but slightly less so than killing off services .

                                To my knowledge all snapvol.cfg's of the appstacks are being merged together (as sabian says here) and if you have 1 appstack that every user gets it would be enough to change the snapvol.cfg on that one appstack.

                                 

                                Regarding the order in which you assign appstacks. I believe (not 100% sure) that indeed the last one assigned is being attached the last and thus "winning" in regards of applications installed within an appstack. Let's say one appstack has Chrome 50 and the other Chrome 60 (yes I know, very old but just for the reference ) the one that is being assigned last will win because both applications are in the same location.