Performance problems in linked clone VMs with any AppStacks attached

[DanVM99]

For some time now we've been finding that certain tasks in our linked clones seem to be taking longer than they should.

Printing is the biggest issue. Sending print jobs takes much longer with an AppStack attached than without. Some of our users are finding that printing from apps such as Word or Adobe Reader (both installed in our Gold Image) to our Canon MFDs or HP printers can take over a minute for the job to be spooled. Print queues are shared out from a printer server and delivered via UEM printer mappings to the non persistent linked clones. Drivers are added to the Gold Image by adding the print queues and then subsequently removing them (leaving the driver behind). If we remove any AppStack assignments then printing is snappy and takes around 15 seconds from clicking 'print', choosing the printer and pressing 'OK' to the job being spooled. This issue occurs with all of our AppStacks. We're using AppVols v2.18 and our apps have been captured using the latest 2.18 template.

We also see terrible performance when trying to browse event logs in event viewer on the clones. Clicking the 'System' results in a busy icon appearing at the top log level and a wait of around 5 minutes before the resulting events are populated in the window. Again, removing AppStacks from the user results in the event viewer behaving normally, listing events within about 15 seconds.

I can't help but feel that overall performance is being degraded by AppVolumes in our environment. Everything "feels" that bit more responsive when we don't have any AppStack attachments. We're using Windows 10 1903 x64 with 6GB RAM and 2CPUs assigned to our clones. I've logged a ticket with Support but if we can't improve things then we're likely to have to look at putting all our Apps in the Gold Image and drop AppVols.

Is anybody else out there seeing behavior like this??

[DanVM99]

For anyone out there who might see this we've also now found that removing the "virtualize registry" lines from the snapvol.cfg in any of our AppStacks has produced noticeable improvements in both the speed of sending print jobs and using the event viewer. Not sure this is a viable configuration though, or even what the virtualize registry lines are responsible for...

[sjesse]

I'd be surprised if a lot of programs work with that setup I'm pretty sure the virtualed_resistry settings redirect the expected registry entries to where the registry hives are mounted from the app stack.

[DanVM99]

Thanks sjesse

We were thinking the same to be honest. We've also noticed that there always tends to be an extra 'SnapVolumes-####' hive mounted in the registry on our clones. For example, if we have one single App assignment/attachment there will be 2 x 'SnapVolumes-###' Reg keys under HKLM. If we had three stacks assigned then there would be 4 of these reg keys. There seems to be a large amount of sub keys and data within them too as when exported they tend to be several hundred MB. Are you able to comment on if your environment looks like this at all?

[sjesse]

Yes, all environments should, if you check C:\snapvolumestemp, the ids match.

and if you look in the mount point and look for hidden files and hidden operating system files the registry hives are the snapvol.dat

Thats why I'm pretty sure removing them will break things.

[Ray_handels]

The extra snapvol hyve I believe is for the delta disk. It keeps track of stuff that changes and need to be removed after logoff if you don't have a writable volume.

And if you were to remark out the virtualize_registry my guess is that close to none application will work within Appvolumes. If you remark something out the Appvolumes filterr driver will not touch that specific part of Windows. We for example excluded almost all TEMP folders (see other topic) in the writable and applications do start up quicker but at the end you will always miss some stuff you need to virtualize. It's a matter of finding the perfect balance for your environment.

[DanVM99]

Thanks Ray, at least we know this behaviour is "by design". What I can't quite understand is why those registry hives contain so much data. When we export them they end up being several hundred MB and contain around 2 million lines of registry in exported form. The result of this seems to be that operations that "parse" the registry such as eventvwr or printing seems to take much longer than they do without any AppStacks attached. Printing a basic one word test doc from MS Word takes around 45 seconds to the point it spools. Without appstacks this process takes just 19s.

We always believed that the capture process would be intelligent enough to only capture changes to the registry during the app install and not capture what appears to be essentially an entire copy of the registry minus a couple of arbitrary exceptions.

Out of interest what's your printing/event viewer performance like?

[Ray_handels]

We see the same behaviour in the Event Viewer but to be honest I don't use it that much as it is a stateless machine so no much usefull info inthere anyway. Normally you need the eventviewer to check what went wrong and normally the machine has been cleaned by the time you want to look up that info in eventviewer.

The reason why it is that big is due to the fact that it merges all registry keys of the machine in those hyves. If you were to browse to the location c:\snapvolumestemp where it creates the mount point you would see that it does not only contain those folders that are created or are present within the appstack, it shows all available folders on your system. There are options to look into the folders or registry hyves that are only available within that appstack but you would need to bypass the Appvolumes Filter driver for that.