11 Replies Latest reply on Jan 18, 2019 4:08 AM by Ray_handels

    Long login times with writable enabled on Instant Clones

    GTO455 Enthusiast

      Hey Folks,

       

      I'm having an issue with app Vols and hope someone else has seen something similar and can help out.

       

      Environment

      • ESXi 6.5
      • App Volumes 2.14
      • VMware Horizon 7.5 (Instant Clones)
      • VMware UEM 9.4
      • Windows 10 Enterprise Build 1709

       

      When we have writable volumes enabled, we're seeing excessive login times upwards of 2 minutes, and get worse with subsequent logins.

       

      First issue is that reviewing the svservices.log, it appears the enabling of the Windows Firewall by the App Vol agent takes an excessive amount of time. If I run the command from an elevated CMD prompt  (within the VM after it has loaded), it takes close to 50 seconds to run, but runs successfully.

       

      This is an example of the firewall launch and completion times

       

      [2018-09-21 22:09:27.158 UTC] [svservice:P1600:T1328] Successfully launched: C:\WINDOWS\system32\cmd.exe /c del /q /f "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" 2>NUL && netsh advfirewall export "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" && netsh advfirewall import "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" (wait -1 ms)

       

      [2018-09-21 22:10:54.641 UTC] [svservice:P1600:T1328] ExitCode 0. Finished waiting for "C:\WINDOWS\system32\cmd.exe /c del /q /f "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" 2>NUL && netsh advfirewall export "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" && netsh advfirewall import "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg"" (WaitStatus 0)

       

       

      The second issue  I have found that if I delete the writable, and recreate it, the login time  decreases dramatically. Subsequent logins after the creation are better, but then they get increasingly worse. It's only until I delete the writable again that things get better.

       

      Has anyone seen anything like this? I had opened a call with VMware support, but they havent been able to solve the problem. Any help you can provide would be much appreciated!

        • 1. Re: Long login times with writable enabled on Instant Clones
          bclyde Novice

          Hello,

           

          We we are seeing exactly the same issue, pretty much down to the letter too. Except we are using Linked Clones.

           

          Did you find an answer to this? Is it a bug in 2.14.0?

           

          Barry

          • 2. Re: Long login times with writable enabled on Instant Clones
            SAsselin007 Novice
            vExpertVMware Employees

            There might be something that you could try.

             

            You could try to add a reverse replication delays in the client machine.

             

            1. In the base image, open regedit and go to this path HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\svdriver\Parameters
            2. create a DWORD value DelayRegistryReverseReplication and set it to 180
            3. Shutdown and use this image as base image for the clones
            • 3. Re: Long login times with writable enabled on Instant Clones
              bclyde Novice

              Hey All,

               

              i managed to find a solution for this issue.

               

              The issue, well - it isn't really an issue at all if the think long enough about it, App Volumes is just doing what it is told to do; its not App Vols fault that your capturing heavy apps :-). Whenever you capture apps in an appstack you also capture their security profile as well. if you have a large appstack, like i did, circa 100GB. Then each application contributes to the size of the Appstack firewall profile. When you go to login as a user to a desktop App Volumes first visualises the OS, then imports the security profile from the appstack to the local desktop. This import/export process can stall logon for up to 40-50 seconds.

               

              If you are in a position where your internal Linked or Instant Clones have their domain firewalls down by default then i see this as being a cumbersome task.

               

              To stop the delay i did the following:

               

              1) update and mount the problematic Appstack on your App Volumes provisioning machine (Use the VMware Console to do this not RDP as connections with drop later)

              2) Open an elevated command prompt and run "netsh advfirewall reset" (this blows all the security config away from the firewall and returns it to default)

              3) run "netsh advfirewall set allprofiles state off" (this takes all the firewalls down)

              4) close the capture and return the Appstack back to the App Volumes Manager.

              5) Delete the user existing writable volume (if they had one) and recreate fresh

              6) Assign a user to the new Appstack and login (remember to unassign the original appstack before logging in with the new one)

               

              I hope it works for you guys. If however you need to have your firewalls on internally then you need to live with this delay as it needs to copy down to the desktop somehow.

              1 person found this helpful
              • 4. Re: Long login times with writable enabled on Instant Clones
                SimonPetrikov Novice

                I'm experiencing the exact same thing and have been troubleshooting for the last 2 days however, I don't have a large AppStack like you do.

                One thing I did do during my troubleshooting was unassign an AppStack so that the user would only have the Writable being attached but I still saw the logon times increase.

                Perhaps the writable was still retaining the data from the AppStack?

                I'll have to give your resolution a try in my environment and see how it works for me.

                 

                My Environment

                • ESXi 6.5
                • App Volumes 2.14
                • VMware Horizon 7.5 (Instant Clones)
                • VMware UEM 9.5
                • Windows 10 Pro Build 1709
                • 5. Re: Long login times with writable enabled on Instant Clones
                  SimonPetrikov Novice

                  Seems that I'm getting this even without an AppStack assigned to a user.

                  • 6. Re: Long login times with writable enabled on Instant Clones
                    Ray_handels Master
                    Community WarriorsvExpert

                    We see the exact same issue but ours is related to the writable. What happens is that the logon goes quite well until it should release the desktop. Then you see the VMLM in the bottom left corner waiting for the logon to finish correctly. When removing the writable volume or recreating it the problem is instantly fixed. Off course this isn;t an option for this. We should be possible to remove the firewall rules in some way. I will raise a call with VMware to see if they can help.

                     

                    Logging of svservice shows this

                     

                    [2019-01-08 13:19:43.781 UTC] [svservice:P1728:T3364] RunCommandLineFromService: Launching CommandLine C:\Windows\system32\cmd.exe /c del /q /f "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" 2>NUL && netsh advfirewall export "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" && netsh advfirewall import "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" (CreationFlag 134217760, wait -1 ms)

                    [2019-01-08 13:19:43.782 UTC] [svservice:P1728:T2636] User has not configured for checking of Windows Update Service status

                    [2019-01-08 13:19:43.789 UTC] [svservice:P1728:T3364] Successfully launched: C:\Windows\system32\cmd.exe /c del /q /f "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" 2>NUL && netsh advfirewall export "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" && netsh advfirewall import "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" (wait -1 ms)

                    [2019-01-08 13:21:26.692 UTC] [svservice:P1728:T3364] ExitCode 0. Finished waiting for "C:\Windows\system32\cmd.exe /c del /q /f "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" 2>NUL && netsh advfirewall export "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg" && netsh advfirewall import "C:\Program Files (x86)\CloudVolumes\Agent\Logs\cvfirewall.cfg"" (WaitStatus 0)

                    [2019-01-08 13:24:34.203 UTC] [svservice:P1728:T1824] Received SERVICE_CONTROL_INTERROGATE

                     

                    As you can see the command takes about 2 minutes to complete. In this time the user only sees a black screen and that's it.

                     

                    If anyone has a solution please feel free to share ..

                    • 7. Re: Long login times with writable enabled on Instant Clones
                      sjesse Master
                      vExpert

                      While I don't use writeable to test, I have been testing pulling vlmn from our images, and logon times have improved. This is with 7.4, to do it correctly you need to remove the vlmn helper from the userinit registry key.

                       

                      VMware Knowledge Base

                      • 8. Re: Long login times with writable enabled on Instant Clones
                        Ray_handels Master
                        Community WarriorsvExpert

                        Hey sjesse,

                         

                        Thanks for your reply. We do see that VMLM is waiting for the logon proces to end but is it not the actual culprit of the issue. Everyone has VMLM enabled here but just a few users have the black screen error. We once did remove VMLM and it did help a few seconds but that was about it. Also, we really like the information we receive from it and because we only have 1 image this would mean noone would have it anymore.,

                         

                        I have raised a call with VMware and am waiting response.

                        • 9. Re: Long login times with writable enabled on Instant Clones
                          Natestack Novice

                          We had a similar issue when attaching a writable it would take 2 to 4 mins to log in.

                           

                          We also couldn't have more than 13 app stacks assigned to the user without appvol going into melt down and not assigning any stacks to the users really annoying if they have more than 13 app stacks anyway.

                           

                          Solution for us was to remove

                           

                          Devices.hotplug false from from the VM issue resolved but now people can remove nic etc so trying to work that one out.

                           

                           

                          Hope that helps you.

                          • 10. Re: Long login times with writable enabled on Instant Clones
                            sjesse Master
                            vExpert

                            What are your logon times with 13 appstacks now? I'm just curious because we are designing ours to limit to 5 at the most. Its a byproduct of an older bug, I'm just hesitant to try using that many appstacks.

                            • 11. Re: Long login times with writable enabled on Instant Clones
                              Ray_handels Master
                              vExpertCommunity Warriors

                              We don't have 13, a max of 10 but to be honest logon times are pretty decent. What I commonly see is that it adds up 1 to 2 seconds to the logontime per appstack because of the processing of the batch files stored on it.

                              And with W10 logon times are that long that it doesn't matter that much if you have more appstacks. Also when logged in we don;t seem to notice a lot of slowdowns due to a bunch of appstacks attached.

                               

                              I do believe though that 10 is still a best practice as stated in official documentation right?