1 2 3 4 5 6 Previous Next 124 Replies Latest reply on Jul 2, 2019 12:04 AM by schepp Go to original post
      • 45. Re: UEM hanging/delaying randomly.
        ijdemes Expert
        vExpert

        BTW, the WPR trace is from the VDI (FlexEngine) side.

        • 46. Re: UEM hanging/delaying randomly.
          UEMdev Master
          VMware Employees

          Thanks, Ivan! Downloading it as we speak – I'll have something to do during the Easter long weekend :-)

          • 47. Re: UEM hanging/delaying randomly.
            Ray_handels Master
            Community WarriorsvExpert
            Does the delay occur consistently at logon? Would you be able to collect a ProcMon trace of a delayed logon?

            No unfortunately it does not. Also, my colleague who does most with this has a week off. We are trying at creating a script to check all log files to see who has the gaps and at what time does this happen. Hoping to get some more info using that but it won't be until next week before we get that going.

            And getting a procmon trace of a delayed logon is quite hard. Because we cannot reproduce it every time we should need to get that procmon running at every logon which stalls the logon and we are using linked clones so it ain't that easy to let procmon run at logon.

            • 48. Re: UEM hanging/delaying randomly.
              UEMdev Master
              VMware Employees

              Hi Ray_handels,

               

              Well, it's good that your users aren't consistently confronted with slow logons, I guess :-) I fully understand the difficulty of catching random delays "in the act" – hope your log analysis can find some pattern after the fact.

               

              In the meantime, I'm keeping busy with ijdemes's WPR trace. Looking very similar to the one that Sabian0309 provided.

              • 49. Re: UEM hanging/delaying randomly.
                Sabian0309 Novice

                I'm still lurking on this .  So i just sent this on the case i am working on with this issue:

                 

                I’m open to suggestions, it does appear to be oplock related.

                 

                To recap, things that were done to the file server:

                 

                Set-SmbServerConfiguration -EnableLeasing $false

                 

                Set-SmbServerConfiguration -EnableOplocks $false

                 

                HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters

                Name : OplockBreakWait

                Type : REG_DWORD

                Value : 10

                 

                However, this I see this on the file server when an issue happens:

                Then 19 seconds later:

                This matches the delay in the logs

                 

                2019-04-23 09:37:06.491 [DEBUG] Read 13 entries from profile archive (size: 66625536; compressed: 8536205; took 493 ms; largest file: 31457280 bytes; slowest import took 185 ms)

                2019-04-23 09:37:25.428 [INFO ] Importing profile archive 'Internet Explorer.zip' (\\jfv-vm-fs2\UEM Profiles\XXXX\archives\Windows Settings\Internet Explorer.zip)

                 

                Now a normal session where this delay is not experienced(no oplock break request):

                • 50. Re: UEM hanging/delaying randomly.
                  Sabian0309 Novice

                  UEMdev,

                   

                  Does UEM do anything funny with exports where it would lock the configuration files as its processing?  As seen in the previous post, the ini file has an oplock where a break is requested.  I've captured earlier in the process, and see a fsctl_request_batch_oplock on the file that causes the delay from another session, guessing from an export of another user as shortly afterwards i see a similarly labeled .tmp then .zip being written:

                   

                   

                  Reading up on the batch uplock, it appears this is requested from the application

                  • 51. Re: UEM hanging/delaying randomly.
                    UEMdev Master
                    VMware Employees

                    Hi Sabian0309,

                     

                    I was hoping you were still lurking on this thread, which is why I @'ed you in my previous post :-)

                     

                    Do you see any change in those 18.9s delays if you tweak that OplockBreakWait registry value?

                     

                    That OPLOCK BREAK in in the file server's ProcMon trace is interesting, as is the subsequent FSCTL_REQUEST_BATCH_OPLOCK (as UEM sure does not send that...) ijdemes, do you see that in your lab, too?

                    • 52. Re: UEM hanging/delaying randomly.
                      Sabian0309 Novice

                      This last weekend i had an outage window where i tweaked the oplockbreakwait setting (and attempted to disable oplocking via powershell), however it did not alter the behavior.  I know with SMB2+ many of the configurations within lanmanserver\parameters are no longer valid, so that may be the case.

                       

                      I'll still poke around some more and see what i can figure out .

                      • 53. Re: UEM hanging/delaying randomly.
                        UEMdev Master
                        VMware Employees

                        Hi Sabian0309,

                         

                        It looks like I missed your "batch oplock request during previous export" post yesterday... Fortunately, you'd also reported that to VMware support, who notified me :-)

                         

                        UEM does not request any oplocks, but I'll try to reproduce this behavior.

                        • 54. Re: UEM hanging/delaying randomly.
                          ijdemes Expert
                          vExpert

                          Hi UEMdev,

                           

                          I will rerun the test with a procmon on file server side and check for oplocks.

                          • 55. Re: UEM hanging/delaying randomly.
                            JohnTwilley Enthusiast

                            I had the Storage Team disable OpLocks on our Isilon SMB shares for UEM.  (Both User Profiles & Configuration)

                            So far, I really think it has helped!  It is quick and easy from the Isilon management console.  I'd share the commands...but I don't have them.

                            You can give them this from EMC Support:

                                          For SMB related inquiries, please provide the following commands output:

                            isi smb shares view YourShareNameHere | grep –i oplock

                            isi_gconfig registry.Services.lwio.Parameters.Drivers.srv.smb2.EnableLeases

                             

                            Side Note:

                            You can test with the following Client registry setting to disable Opportunistic Locks from a client PC.

                             

                            Turn Off Client requests for OpLocks for SMB

                             

                            [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\mrxsmb\Parameters]

                            "OplocksDisabled"=dword:00000001

                             

                            This is not what I’d like to use going forward, as it will turn off OpLocks for All SMB shares, not just the Isilon ones.

                            • 56. Re: UEM hanging/delaying randomly.
                              ijdemes Expert
                              vExpert

                              Hi Sabian0309,

                               

                              I see the same happening in my trace on the file server (didn't see it the first time, as I forgot to enable advanced logging ).

                               

                              <click pictures for the full lines>

                               

                              I did try some settings with Oplocks on the file server and client (VDI) side. But without any positive result so far.

                              • 57. Re: UEM hanging/delaying randomly.
                                ijdemes Expert
                                vExpert

                                Hi JohnTwilley,

                                 

                                Thanks for your suggestion. I'm not sure if I checked the client setting you mentioned, as I tried a lot of options. But I will double check again and respond here about the result.

                                • 58. Re: UEM hanging/delaying randomly.
                                  ijdemes Expert
                                  vExpert

                                  I changed the settings you mentioned. Both the OplocksDisabled reg key and settings on the Windows file server, like OplocksDisabled and Leases. But both of these settings result in the same trace output. I still see Oplock messages.

                                  • 59. Re: UEM hanging/delaying randomly.
                                    UEMdev Master
                                    VMware Employees

                                    Hi ijdemes and Sabian0309,

                                     

                                    I'm setting up a new environment to run some more tests, and I just noticed the following:

                                    I'm not an IT Pro but just a lowly developer, so I don't know whether it's relevant, but I was wondering how your shares are set up w.r.t. these caching/offline settings.

                                    1 2 3 4 5 6 Previous Next