Forgot another question : I can`t capture boot logging because the c:\ drive is refreshed after a reboot. (sort of readonly when a appstack is attached?) Tools i`ve been trying store files during capture on c drive . Can`t change this.
Anyone knows how i can do this?
- Process Monitor (procmon)
- Microsoft Windows Performance Toolkit- Xperf
- WPR (Windows performance recorder)
We use an agent-less scanner (McAfee Move) but I do understand that you cannot go that route.
How many CPU's do you have? What happens if you try to add some more CPU's? You will need at least 2 vCPU's, otherwise Appvolumes and Symantec are both fighting for resources during logon.
And what exclusions do you have? Did you try adding the c:\snapvolumestemp\Mountpoints as an exclusion??
We use 2 vCPU`s and 4 GB memory. We`ve tested with 4vCPU`s and 8GB memory. No change in login performance.
We have the following exclusions :
C:\PROGRAM FILES (X86)\BIGFIX ENTERPRISE\BES CLIENT\BESCLIENT.EXE
C:\PROGRAM FILES\VMWARE\VMWARE VIEW\AGENT\BIN\V4PA_AGENT.EXE
C:\PROGRAM FILES\IMMIDIO\FLEX PROFILES\FLEXSERVICE.EXE
%[PROGRAM_FILES]%Immidio\Flex Profiles\Flex+ Self-Support.exe
That should be enough looking at the exclusions. I would suggest using more then 4GB for a VDI machines but that has got nothing to do with virusscanning and slow logon times, just a best practice.
To be honest I don't now how to fix this. Would it be an option to try and run Windows Defender just as a reference?? It might be the way Symantec works maybe? You will always receive a logon penalty from virus scanners but it should not be this much.
What happens if you try user-assigned instead of computer-assigned?
We also have SEP 14.2.3335.1000 running in our environment with the same exlusions set on our SEPM servers and login times are a minute or less with a couple appstacks applied.
VMs are currently 2CPU/6GB RAM and we have machines on both Windows 10 1709 and 1809 LTSC.
We are seeing the same issue here. Removing SEP altogether has shown a x5 increase in almost anything we use. Logon, AppVolumes attachment, AppV launch and cache, general usability are all dramatically improved.
Are there any additional settings over Application and Device control that can be set to improve performance?
I'm sorry to say that Hmunning, but I'm happy someone else is facing the same problem, as I was starting to become crazy and feeling alone !
I'm fighting with this issue for more than a year (almost 2 actually) on Windows 7 machines. On Windows 10, I haven't tested a lot for now as we purchased licences recently.
By the way, Hmunning, you didn't tell us what Windows version you're using.
I currently use the following versions :
- Windows 7 SP1 x64
- AppVolumes : 2.18
- Horizon View Agent 7.6
- SEP 14.2.4814.1101
- UEM (now called DEM) 9.9
Interesting fact : I notice this problem only appears after I apply Microsoft Security KB4056897 or later (and of course, with SEP agent installed and active and AppStacks mounted)
This means the problem is not only with SEP + AppVolumes, but SEP + AppVolumes + MS Updates (starting january 2018 and all the Intel security breaches fixes).
If I remove ANY ONE of these 3 elements, everything works well.
Until now, no Monthly security updates from Microsoft has solved anything.
I had a case at VMware and another one at Symantec, but none of them solved the problem.
I sent them dumps, videos of the user experience with and without the problem, logs... We had WebEx sessions, phone call, emails...
After months, we had this answer from VMware :
Our backline engineers provided feedback about the procmon log files provided. They confirmed that the issue is related with a performance drop down that is making the access to registry key take longer. They could confirm that Symantec is the component that is making our driver work slow.
They suggested you contact Symantec and confirm if they have a way to solve the performance drop and/or to exclude App Volumes driver.
We were about to organize a call between VMware and Symantec, but finally we closed the cases, unresolved, hoping for a better experience with Windows 10...
And now I'm working on Windows 10 migration, and I still have to test AppVolumes and SEP... And still hoping for this issue to not rise!
What are the case numbers with VMware and Symantec, the customer above is using Windows 8.1 and had similar issues with Windows 10 as far as I know.
Here are the cases numbers :
VMware : 18862278007 and 18997393111
Symantec : 15162388
We actually had to ditch SEP altogether because of this specific issue. We never couldn’t get them to play nicely together. We saw around 1 minute per simple AppStack when SEP was involved.
That's what I'm fighting for, but I doubt it will be possible unfortunately.
Just out of curiosity, what did you replace it with ?
It was between Endgame and Defender; we ended up going with Endgame. We also have NSX doing micro-segmentation, which can help if you're in an AV debate with InfoSec.
OK. We unfortunately don't use NSX, and it's not supposed to happen soon...
When you say you were not able to make SEP work well with AppVolumes, what were the paths you went on ? Did you get help from VMware and Symantec, or did you try to solve this by yourself ?
Are there any Support Case that we can refer to ?
I am a little pleased to hear that we are not the only ones with this problem.
After a few months of troubleshooting we have only achieved small results. From the current support calls of Symantec and VMware no rootcause has been found yet. We have delivered many gb of logfiles and videos. Procmon, WPA,XPerf, Wireshark. You name it...
Both Symantec and VMware indicated that no other support calls are registred with this specific problem. Well that was not true, but maybe difficult to find? For now the only thing what is seen is there are extremely high CPU spikes and Symantec scans a lot of registry entries. The filter drivers of Symantec, App Volumes and DEM all want to use their resources probably at the same time. Due to the spectre and meltdown patch the performance degradation in this combination is severe.
By accident we found out that symantec didn't work at all !! Every thing looked fine. The client was healthy from de management server point of view, but stopping and then starting the smc.exe resulted in a crash. A simple EICAR virus test was not even detected !! Through exceptions in the snapvol.cfg we got symantec working properly. We want to have these exceptions validated by VMware. There is a PR opened up for this.
Since Symantec is working now we see better (not optimal) startup times of thinapps in an app stack . Login times unfortunately not. We declared all de collected log files to be unreliable, because the symantec client did not work at all. And so the exceptions and exclusions may not have worked at all. We collected all log files again recently.
I will update you when we hear something back from Symantec or VMware.