WindowServer crashing when running Win10 VM

c_harms · ‎08-08-2020

Hi everyone,

I am using VMware Fusion 11.5 on my MacBook Pro (15", 2018) and for a couple of weeks now, my whole system has become mostly unusable when the VM is running.

The issue emerged when using macOS Catalina 10.15.5, so I do not think it is related to the known issue relating to the memory leak. Since then I have updated to 10.15.6, but the issue remains. I have not yet encountered the memory leak as the system is crashing before. The problem exists in both VMware Fusion 11.5.5 and 11.5.6.

What is happening? When using the VM over time the whole system (both guest OS and host OS) hangs for some seconds every other minute. After some of these freezes (sometimes just a few minutes, sometimes after an hour), everything goes black, I am back at the macOS login screen. When logging in, the VMware is not visible anymore (but still running), so I have to suspend and restart the VM. Then, again, I have a couple of minutes running the VM until the same issue happens again.

Using a freshly booted host and/or guest OS does not make any difference.
The issue only occurs when the VM is running - absolutely no problems, when the VM is turned off or suspended.
It seems to me the issue occurs more quickly when I have many windows open, but I haven't validated this systematically yet.
Deactivating 3D acceleration does not solve this issue.

When looking at Console.app the culprit seems to be macOS' WindowServer (see attached crash report):

Process: WindowServer [288]

Path: /System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/Resources/WindowServer

Identifier: WindowServer

Version: 600.00 (451.4)

Code Type: X86-64 (Native)

Parent Process: launchd [1]

Responsible: WindowServer [288]

User ID: 88

Date/Time: 2020-08-08 13:01:17.161 +0200

OS Version: Mac OS X 10.15.6 (19G73)

Report Version: 12

Bridge OS Version: 4.6 (17P6065)

Anonymous UUID: 083DF09A-B169-7F44-E3EB-1E021FF1CEDC

Time Awake Since Boot: 10000 seconds

System Integrity Protection: enabled

Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Exception Type: EXC_CRASH (SIGABRT)

Exception Codes: 0x0000000000000000, 0x0000000000000000

Exception Note: EXC_CORPSE_NOTIFY

In system.log I see a couple of messages, but I have those mdworker-logs also when the system does not freeze:

Aug  8 12:55:39 MacBook-Pro-3 VTDecoderXPCService[74544]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
Aug  8 12:55:39 MacBook-Pro-3 VTDecoderXPCService[74544]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
Aug  8 12:55:39 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.pid.mdmclient.74575): Failed to bootstrap path: path = /usr/libexec/mdmclient, error = 108: Invalid path
Aug  8 12:55:40 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.pid.backupd.74579): Failed to bootstrap path: path = /System/Library/CoreServices/backupd.bundle/Contents/Resources/backupd, error = 2: No such file or directory
Aug  8 12:56:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.01000000-0700-0000-0000-000000000000[74580]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.10000000-0300-0000-0000-000000000000[74570]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.03000000-0700-0000-0000-000000000000[74583]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:19 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.05000000-0400-0000-0000-000000000000[74584]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:19 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.04000000-0200-0000-0000-000000000000[74586]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:19 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.07000000-0200-0000-0000-000000000000[74110]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:22 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0F000000-0400-0000-0000-000000000000[74582]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:44 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.02000000-0100-0000-0000-000000000000[74588]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:56:48 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.08000000-0700-0000-0000-000000000000[74587]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.09000000-0100-0000-0000-000000000000[74571]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0C000000-0200-0000-0000-000000000000[74581]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.06000000-0100-0000-0000-000000000000[74578]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:49 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.08000000-0000-0000-0000-000000000000[75390]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:49 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0F000000-0500-0000-0000-000000000000[75216]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:49 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.05000000-0500-0000-0000-000000000000[75295]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:57:57 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.06000000-0200-0000-0000-000000000000[75759]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:58:13 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.CloudDocs.MobileDocumentsFileProvider[74229]): Service exited due to SIGKILL | sent by launchd[1]
Aug  8 12:59:06 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0F000000-0600-0000-0000-000000000000[76153]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 12:59:39 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.07000000-0300-0000-0000-000000000000[75296]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:00:22 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.pid.TouchBarServer.410): Failed to bootstrap path: path = /usr/libexec/TouchBarServer, error = 108: Invalid path
Aug  8 13:00:30 MacBook-Pro-3 Microsoft Edge[74537]: BUG in libdispatch client: mach_recv, monitored resource vanished before the source cancel handler was invoked { 0x7fa521424370[source], ident: 73275 / 0x11e3b, handler: 0x106ea92f0 }
Aug  8 13:01:07 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.08000000-0100-0000-0000-000000000000[77924]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:07 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0A000000-0200-0000-0000-000000000000[77980]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:07 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0D000000-0000-0000-0000-000000000000[77981]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:07 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0E000000-0700-0000-0000-000000000000[77979]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:08 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.02000000-0200-0000-0000-000000000000[77978]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:08 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.10000000-0400-0000-0000-000000000000[77976]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:08 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.03000000-0000-0000-0000-000000000000[77975]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.01000000-0000-0000-0000-000000000000[77977]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.06000000-0200-0000-0000-000000000000[77973]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.04000000-0300-0000-0000-000000000000[77974]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0C000000-0300-0000-0000-000000000000[77847]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.05000000-0600-0000-0000-000000000000[76560]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.09000000-0200-0000-0000-000000000000[77972]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.07000000-0400-0000-0000-000000000000[77637]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:09 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0F000000-0700-0000-0000-000000000000[77821]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:12 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.mdworker.shared.0B000000-0200-0000-0000-000000000000[77982]): Service exited due to SIGKILL | sent by mds[195]
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.WindowServer[288]): Service exited due to SIGABRT
Aug  8 13:01:17 MacBook-Pro-3 sessionlogoutd[78459]: DEAD_PROCESS: 252 console
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.ViewBridgeAuxiliary[78461]): Service exited with abnormal code: 2
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.imklaunchagent): This service is defined to be constantly running and is inherently inefficient.
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.coreservices.UASharedPasteboardProgressUI): Unknown key for Boolean: DrainMessagesAfterFailedInit
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.UserEventAgent-LoginWindow): This service is defined to be constantly running and is inherently inefficient.
Aug  8 13:01:17 MacBook-Pro-3 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.user.loginwindow.78463.4294967295): com.apple.universalaccessd (lint): The HideUntilCheckIn property is an architectural performance issue. Please transition away from it.

I have attached the vmware-vmfusion.log file, but could not find any evident problem there as well.

Using Google or searching the communities did not reveal any issue that might be related except for the memory leak bug, but since the problem occured already in the previous version of Catalina I do not think it is related.

Any ideas how to resolve this except for re-installing both guest and host OS?

Thanks

Christopher

scott28tt · ‎08-08-2020

To be clear, your Windows 10 VM runs nested on Hyper-V, in a Windows Server VM which runs on Fusion?

-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog

c_harms · ‎08-08-2020

Um, no, sorry... I have a macOS Catalina host system running Fusion in which the VM runs Windows 10. WindowServer was referring to the process on the host system that crashes according to the console. There is no nested VM - just plain an simple Windows 10 inside Fusion on macOS.

Best

Christopher

scott28tt · ‎08-08-2020

I see what you mean, the WindowServer component of MacOS - silly me!

-------------------------------------------------------------------------------------------------------------------------------------------------------------

Although I am a VMware employee I contribute to VMware Communities voluntarily (ie. not in any official capacity)
VMware Training & Certification blog

wila · ‎08-08-2020

Hi,

Can you attach a vmware.log file from the VM?

Eg. right click on the VM's bundle, then select "Show Package Contents" and grab the vmware.log file in there then attach it to a reply here.

If I'm understanding you correctly then the WindowServer crash happens within the hour? (so we can exclude the kernel memory leak from 10.15.6)

Are there any crash dumps in that folder?

As it is the window manager that is crashing there's a couple of usual suspects.

One as you already identified is the graphics adapter, but other factors might play a role too.

The vmware.log file should give us an idea about any potential configuration issues.

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva

c_harms · ‎08-09-2020

Hi,

I have attached the log file corresponding to the crash occuring yesterday (same as in the uploaded files before). Crash occurred around 13:01

If I'm understanding you correctly then the WindowServer crash happens within the hour? (so we can exclude the kernel memory leak from 10.15.6)

Yes, it is more a short-time issue. When the VM crashes it does so within an hour or one and half max. Often sooner.

Are there any crash dumps in that folder?

No, only log files and the usual suspects.

Hope the log file helps – I couldn't see anything problematic in particular.

Best

Christopher

wila · ‎08-09-2020

Hi,

Long day, so didn't come to this earlier.

Starting from the top, as we're having a window manager crashing anything non standard related to the screen drivers is suspect.

I noticed you have the following settings in your user preferences file (~/Library/Preferences/VMware Fusion/preferences)

mks.enableMTLRenderer = "1"
mks.enableGLRenderer = "0"
mks.gl.allowBlacklistedDrivers = "TRUE"

Please remove all of those.

Especially the hint that suggests to use blacklist drivers can cause all kinds of things.

Why is it in there?

Are you perhaps trying to use an external GPU?

The vmx file of the VM itself also has this line:

mks.enableGLRenderer = "1"

which AFAICR is also not standard and as you can see ... you're now telling the VM to both use openGL _and_ metal for rendering.

I can imagine that alone might trigger an issue.

In addition I have never seen the values "1" and "0" used with those settings, but always "TRUE" and "FALSE", not sure if that matters either.

I'm not saying that the lines above are the problem, but I would remove all these non standard lines and try if the problem still happens in order to exclude that possibility.

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva

c_harms · ‎08-09-2020

Thanks a lot, wila, for taking the time!

Truth be told, I have forgotten about these settings. I have added them after I had issues with warnings and error messages that occurred in the past.

When I had problems with the error that no 3D acceleration was available (message appeared after 10.15.4, I think) I found several posts recommending to make those changes (e.g. https://apple.stackexchange.com/questions/343473/how-to-resolve-no-3d-support-is-available-from-the-... ) and I might have not removed them afterwards.

I have removed these entries in both files and will give it another try. I'll report back when I know if this improved the issue.

Best

Christopher

c_harms · ‎08-10-2020

Good morning,

yesterday I had no crashes and was already super happy to have fixed the issue. Unfortunately, after working for half an hour this morning, I had another crash. I have attached a new vmware.log and WindowServer's crash report. After the logging back in after the crash, I suspended the VM manually (because it is not visible anymore in Unity mode).

Crash occured around 08:34+01:00. Before it crashed, I also had some freezes - meaning, that both Guest and Host system became unresponsive for a couple of seconds. The third freeze resulted in the WindowServer crash.

Both yesterday and this morning I have used the same software within the VM (a data science / ETL software that is heavy on RAM and CPU but does not require any GPU power).

Any new ideas?

Best

Christopher

wila · ‎08-11-2020

Hi,

If I am reading this correctly then you are only getting the issue in unity mode?

Can you try and see if it still happens without using unity mode?

Around the time of the WindowServer crash I see a lot of lines like this one:

2020-08-11T08:34:56.669+01:00| windowThread-3| W003: MKSUIDrawnMKSClient_SendUIDrawnMKSCommand, could not create Mach port to send UI command.

A standard troubleshooting step would be to try and re-install vmware tools.

eg. uninstall VMware Tools via the windows settings -> apps

Then reboot the guest and install VMware Tools again.

I have my doubts on that being the culprit, but it is one thing to try.

If it happened to me I would probably also try to create a log dump.

eg. write down the time it happens (you already are doing that, thanks for that detail!)

Then generate a log bundle with the log command.

sudo log collect --last 3m

which would collect all the logs for the past 3 minutes and store that as a bundle in the folder you are.

This you can then inspect with console and see if that gives any hints.

If none of that helps, then we're down to it being either a hardware issue or something strange happening with VMware Fusion.

You can try the built in apple hardware test to see if that gives something.

For VMware Fusion you can get support via the official support channel (please go to: Fusion Support and open a ticket at File a Support Request)

Hope this helps,

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva

c_harms · ‎08-18-2020

Thanks, wila!

I was able to narrow down the problem. After re-installing VMware Tools it seemed to work well for a while, but the next day I had the same problem again. Using the log collection, I found a couple of suspect log entries, that seem to hint at a problem with the AMD Radeon GPU driver.

When the GUI starts freezing, this messages starts to appear every other minute:

fault    2020-08-18 11:46:28.171314 +0200    kernel    void IOAccelFenceMachine::fence_timeout(IOTimerEventSource *): AMDRadeonAccelerator prodding blockFenceInterrupt

Before the system crashes, this message appears several times directly in sequence until WindowServer notes

default 2020-08-18 11:46:28.096805 +0200 WindowServer GPU Driver for display 0x1b565e29 appears to be hung (5 continuous seconds of unreadiness)

This repeats for several seconds (with increasing number of continuous seconds of unreadiness) with some attempts to restart the GPU intermittently:

error    2020-08-18 11:46:29.041363 +0200    kernel    [1:0:0]: channel 14 DisplayPipe0 is hung! (lastReadTimestamp=0x0001ece7) channelResetMask 0x00000000
fault    2020-08-18 11:46:29.041368 +0200    kernel    void IOAccelEventMachine2::restart_channel(): GPURestartBegin stampIdx=14 type=2
fault    2020-08-18 11:46:29.041409 +0200    kernel    virtual void IOAccelFIFOChannel2::restart(): ring is empty and all finished. Nothing to do.
fault    2020-08-18 11:46:29.041410 +0200    kernel    virtual void IOAccelFIFOChannel2::restart(): GPURestartSkipped stampIdx=14
fault    2020-08-18 11:46:29.041413 +0200    kernel    void IOAccelEventMachine2::restart_channel(): GPURestartEnd stampIdx=14 type=2
fault    2020-08-18 11:46:29.041423 +0200    kernel    void IOAccelEventMachine2::hardwareErrorEvent(): setting restart type to 2 (channel 17)
fault    2020-08-18 11:46:29.041424 +0200    kernel    void IOAccelEventMachine2::hardwareErrorEvent(): GPURestartDequeued stampIdx=17 type=2
error    2020-08-18 11:46:29.041426 +0200    kernel    [1:0:0]: channel 17 event timeout

Similar messages appear repeatedly until the login window appears.

So, it seems to be related to the GPU - however, the problem only occurs when VMware is running (having checked with console running live). 3D acceleration is turned off in the VM and turning it on does not change the problem.

Since video playback is unaffected, I still believe it's a problem with VMware and how it communicates with the internal Radeon GPU.

Any further ideas?

Best

Christopher

wila · ‎08-18-2020

File an official support request... at least then you'll get a developer looking at your report who can determine if it is something in their code or a driver issue.

The only other thing I can suggest is to try the VMware Fusion Tech Preview to see if it also happens there, but I would still open the ticket with VMware in your case.

--

Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva

bluefirestorm · ‎08-18-2020

In the Energy Saver of System Preferences, is the automatic graphics switching disabled?

From the log, it might look like the case that the Radeon went on hibernation to save power and seem to fail to restart.

c_harms · ‎08-24-2020

I tried it both with the option activated and deactivated - no change to the log and to the crashes.

After some further tests, it seems that the problems with the Radeon GPU only occur when running fusion in fullscreen or Unity mode. When running the guest OS only in a single window: No crashes and no GPU faults in the log.

Haven't opened a support ticket yet - need to do this through our IT department as my account is not linked to the license.

All

WindowServer crashing when running Win10 VM