VMware Communities
DMancini_XDS
Contributor
Contributor

Black Screens in Workstation

Instead of working displays, I am getting black/glitched screens in Workstation (guests can still be viewed over RDP):

  • Version:  Workstation 17.5.1 build-23298084,
  • Host:  Windows Server 2019 (Build 17763.5458)  10.0.17763
  • 2x Guests:  Windows Server 2019 (Build 17763.5458)  10.0.17763
22 Replies
DCasota
Expert
Expert

+1 Bump

Same seems to happen on Linux guest os as well,  since 17.5.1.

Updating Photon OS 3.0 vm with photon-update.sh has the same effect on next reboot. The main console is affected, but login e.g. through alt-f2 works. Might be Hyper-v/WSL-related. I have to assemble more findings.

0 Kudos
DMancini_XDS
Contributor
Contributor

I can't say for absolutely certain yet as I'm going to have to wait and watch, but it appears that forcing processor affinity for my VMs seems to have banished this issue.

0 Kudos
DMancini_XDS
Contributor
Contributor

Strike that last - rebooting the host and pulling the VMs from suspension caused both to have untouchable black screens in Workstation.  Still alive and working though, and reachable by RDP.

0 Kudos
batuhandemirdal
Enthusiast
Enthusiast

Hi,

Have you tried safe mode?

Tags (1)
0 Kudos
DMancini_XDS
Contributor
Contributor

No.  Why would "safe mode" have anything to do with this?  And what "safe mode" are you talking about?

0 Kudos
DMancini_XDS
Contributor
Contributor

To add some more observational evidence, it seems like auto-started VM's in general (whether or not they were suspended) boot up with the display and inputs broken/nonfunctional in Workstation.

0 Kudos
bluefirestorm
Champion
Champion

Two things you could try
(1) use Console view (go to View menu - Console view), the Library will disappear but the VM screen will appear as normal resolution as set inside the guest OS.
(2) (this might sound like a bad joke but it isn't a joke) the screenshot looks like "Dark Mode" is enabled, have you tried setting the Preferences of Display Color Theme to "System" instead of "Dark"?

0 Kudos
NateNateNAte
Hot Shot
Hot Shot

So is this host a physical MW2019 server, or a virtual MS2019 server?  It does make a difference.  I'm asking so I can do some more specific research to get to the root cause here. 

0 Kudos
DMancini_XDS
Contributor
Contributor

I am running Windows server 2019 standard on the metal, and both virtual machines are also Windows Server 2019 standard.

0 Kudos
DMancini_XDS
Contributor
Contributor

Toggling console view on and off does nothing. I tried switching the workstation display themes and that did nothing either. It was already set to use system anyway.

 

Allow me to restate that this specifically only happens when a virtual machine is auto started. This issue does not seem to afflict virtual machines that I start manually. It's like something is not getting hooked properly when they are Auto started.

0 Kudos
bluefirestorm
Champion
Champion

So the vmrun launches into the background without gui and then the VM(s) is(are) opened through the Workstation Pro UI and black screen occurs.
Is the user account that was used to launch the vmrun the same user account that is experiencing the black screen?
Try launching Workstation Pro with "Run as admin" and then open the VM(s) that were launched using vmrun.

 

0 Kudos
DMancini_XDS
Contributor
Contributor

The virtual machines are not being started by vmrun, they are being started by the vmware-autostart.exe service as configured / instructed through the Workstation UI.  The service is set to run under my user account.

Running Workstation "As Administrator" doesn't seem to have any effect or make the display work.

0 Kudos
bluefirestorm
Champion
Champion

Is "3D acceleration" enabled on the VMs? My guess it is not.
I can only suspect that when the autostarted VMs were opened from the GUI, it was expecting to have mksSandbox.exe process (aside from vmware-vmx.exe) running for that VM, and couldn't find it and then just went black. Maybe it is written in the vmware.log.
The mksSandbox.exe would handle mks, probably stands for mouse, keyboard, screen. When 3D acceleration is enabled, mksSandbox.exe would run alongside vmware-vmx.exe per VM.
The separate mksSandbox.exe process per VM can be disabled. The processes that were sandboxed into mksSandbox.exe will run inside vmware-vmx.exe instead (just like version 15.x and earlier).
Use a Command Prompt with Admin edit %PROGRAMDATA%\VMware\VMware Workstation\config.ini and add the following lines

mks.requireISBRenderer = "FALSE"
mks.enableISBRenderer = "FALSE"

This setting will affect all VMs.

This is the only suspect that I have why an autostarted VM might turn up black when opened from GUI. I got no other ideas at the moment.

0 Kudos
DMancini_XDS
Contributor
Contributor

Here is a snippet of the vmware.log (several references to mks failures):

 

2024-03-17T04:09:13.225Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 2.
2024-03-17T04:09:13.408Z In(05) mks SWBWindow: Window #0 validation failed: no valid host window or host surface.
2024-03-17T04:09:13.408Z In(05) mks SWBVmdb: Destroy SWB Window Id #0 because an invalid MKSWindow definition is received from UI over VMDB.
2024-03-17T04:09:13.409Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 1.
2024-03-17T04:09:13.409Z In(05) mks SWBWindow: Window #1 validation failed: no valid host window or host surface.
2024-03-17T04:09:13.409Z In(05) mks SWBVmdb: Destroy SWB Window Id #1 because an invalid MKSWindow definition is received from UI over VMDB.
2024-03-17T04:09:13.409Z In(05) mks GDI-Backend: stopped by HWinMux to do window composition.
2024-03-17T04:09:13.410Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 0.
2024-03-17T04:09:14.466Z In(05) mks SWBWindow: Number of MKSWindows changed: 1 rendering MKSWindow(s) of total 1.
2024-03-17T04:09:14.467Z In(05) mks GDI-Backend: successfully started by HWinMux to do window composition.
2024-03-17T04:09:14.467Z In(05) mks MKS Win32: Failed window creation for MKSWindowId=0: Invalid UI Window
2024-03-17T04:09:14.467Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=0)
2024-03-17T04:09:14.468Z In(05) mks MKS-HWinMux: Started GDI presentation backend.
2024-03-17T04:09:14.470Z In(05) mks SWBWindow: Number of MKSWindows changed: 1 rendering MKSWindow(s) of total 2.
2024-03-17T04:09:14.470Z In(05) mks MKS Win32: Failed window creation for MKSWindowId=1: Invalid UI Window
2024-03-17T04:09:14.471Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=1)
2024-03-17T04:09:15.980Z In(05) mks MKS-VMDB: VMDB requested a screenshot
2024-03-17T04:09:15.980Z In(05) svga MKSScreenShotMgr: Taking a screenshot
2024-03-17T04:09:16.345Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 2.
2024-03-17T04:09:16.538Z In(05) mks SWBWindow: Window #0 validation failed: no valid host window or host surface.
2024-03-17T04:09:16.538Z In(05) mks SWBVmdb: Destroy SWB Window Id #0 because an invalid MKSWindow definition is received from UI over VMDB.
2024-03-17T04:09:16.538Z In(05) mks MKSCursorPosition: Destroying the current grab window
2024-03-17T04:09:16.544Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 1.
2024-03-17T04:09:16.544Z In(05) mks SWBWindow: Window #1 validation failed: no valid host window or host surface.
2024-03-17T04:09:16.544Z In(05) mks SWBVmdb: Destroy SWB Window Id #1 because an invalid MKSWindow definition is received from UI over VMDB.
2024-03-17T04:09:16.544Z In(05) mks GDI-Backend: stopped by HWinMux to do window composition.
2024-03-17T04:09:16.545Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 0.
2024-03-17T04:09:17.276Z In(05) mks SWBWindow: Number of MKSWindows changed: 1 rendering MKSWindow(s) of total 1.
2024-03-17T04:09:17.276Z In(05) mks GDI-Backend: successfully started by HWinMux to do window composition.
2024-03-17T04:09:17.276Z In(05) mks MKS Win32: Failed window creation for MKSWindowId=0: Invalid UI Window
2024-03-17T04:09:17.277Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=0)
2024-03-17T04:09:17.277Z In(05) mks MKS-HWinMux: Started GDI presentation backend.
2024-03-17T04:09:17.280Z In(05) mks SWBWindow: Number of MKSWindows changed: 1 rendering MKSWindow(s) of total 2.
2024-03-17T04:09:17.280Z In(05) mks MKS Win32: Failed window creation for MKSWindowId=1: Invalid UI Window
2024-03-17T04:09:17.280Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=1)
2024-03-17T04:09:18.401Z Wa(03) mks SOCKET 13 (-1) AsyncNamedPipeRecvCallback: failed to get overlapped result: 109.
2024-03-17T04:09:18.401Z In(05) mks MKSControlMgr: MKSControl Remote Disconnect: socket closed.
2024-03-17T04:09:18.401Z Wa(03) mks MKSResponse: Error: (1084)
2024-03-17T04:09:18.401Z In(05) mks MKSControlMgr: MKSResponse error
2024-03-17T04:09:18.401Z In(05) mks MKSControlMgr: Dropping MKSControl error due to prior unresolved error.
2024-03-17T04:09:18.401Z Wa(03) mks MKSControlMgr: MKSControl Error: Disconnecting from UI
2024-03-17T04:09:18.402Z In(05) mks MKSControlMgr: disconnected
2024-03-17T04:09:19.265Z In(05) vmx SOCKET 12 (2848) recv error 10054: An existing connection was forcibly closed by the remote host
2024-03-17T04:09:19.265Z In(05) vmx Vix: [mainDispatch.c:2813]: VMAutomation: Connection Error (1) on connection 3.
2024-03-17T04:09:19.266Z In(05) vmx VmdbPipeStreamsOvlError Couldn't read: (10054) An existing connection was forcibly closed by the remote host.
2024-03-17T04:09:19.266Z In(05) vmx VmdbCnxDisconnect: Disconnect: closed pipe for pub cnx '/db/connection/#18/' (-32)
2024-03-17T04:09:19.268Z In(05) vmx VmdbDbRemoveCnx: Removing Cnx from Db for '/db/connection/#18/'
2024-03-17T04:09:19.268Z In(05) mks MKSCursorPosition: Destroying the current grab window
2024-03-17T04:09:19.271Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 1.
2024-03-17T04:09:19.271Z In(05) mks GDI-Backend: stopped by HWinMux to do window composition.
2024-03-17T04:09:19.271Z In(05) mks SWBWindow: Number of MKSWindows changed: 0 rendering MKSWindow(s) of total 0.
2024-03-17T04:10:00.022Z In(05) vcpu-6 DISKUTIL: scsi0:0 : capacity=268435456 logical sector size=512
2024-03-17T04:10:00.023Z In(05) vcpu-4 DISKUTIL: scsi0:0 : capacity=268435456 logical sector size=512
2024-03-17T04:10:59.990Z In(05) vcpu-4 DISKUTIL: scsi0:0 : capacity=268435456 logical sector size=512
2024-03-17T04:10:59.992Z In(05) vcpu-5 DISKUTIL: scsi0:0 : capacity=268435456 logical sector size=512

0 Kudos
DMancini_XDS
Contributor
Contributor

Also I added those two lines to the config, to no avail.  This seems like a problem with the latest version of Workstation, since I have NEVER had this issue with any previous version.

See image for the nonsense it dumps on the screen when I try to simply resize the window.

 

20240316-2126.PNG

0 Kudos
bluefirestorm
Champion
Champion

I think the autostart feature is new to 17.5. It is available only in Windows hosts. Don't know how different it is from starting vmrun -nogui manually but there is the convenience factor of not having to login to the host. On the surface it looks a bit similar to the Shared VMs feature which was killed off in 16.x, resurrected somewhat and killed off again in 17.x

I use mainly 16.2.5 on an Ubuntu host; but don't see these sort of problems when I close a running VM window and keep it running in the background and then reopen it. VMware Workstation on Linux is a different beast though when it comes to the graphics layer.

I forgot to mention that the VMs have to be restarted after adding those two lines in the config.ini so that at least the vmware-vmx.exe explicitly starts without trying to enable ISB rendering. It does not look promising though as the vmware.log snippet showed multiple times to start GDIPresentation; so vmware-vmx.exe is probably already aware there is no mksSandbox process to look for.

2024-03-17T04:09:14.467Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=0)
2024-03-17T04:09:14.468Z In(05) mks MKS-HWinMux: Started GDI presentation backend.
2024-03-17T04:09:17.277Z In(05) mks MKS-HWinMux: Failed PreDefineWindow (MKSWindowId=0)
2024-03-17T04:09:17.277Z In(05) mks MKS-HWinMux: Started GDI presentation backend.

If you can install a cheap and cheerful Nvidia graphics card into the R630 it might make a difference; even an entry level Nvidia graphics card is better than none though for 17.5.x the minimum for Nvidia is Maxwell Gen 2 to use DX12; so perhaps a Pascal GT1030 or its entry level Quadro equivalent; older than that require additional vmx configuration. An extra monitor won't be required as the GPU can still function as render device without having an actual display. With Nvidia GPU, it won't be GDI presentation backend; as it will have to be either DX12 or DX11 presentation back end, depending on the GPU used on the Windows host.

0 Kudos
DMancini_XDS
Contributor
Contributor

I see.... not really wanting to add to the cost at this point, since i just sprung for a license for this software and it worked on this machine before.

I did think of restarting the VMs after changing that config file; it didn't help.

I just added a couple vmrun commands into my startup apps for the main (and only) login-user account, since I already have a tool I use to auto-logon-and-lock the machine anyway.  That works just fine (and the display/input works when opening Workstation).

Would have been nice if VMWare had delivered a fully working auto-start feature as part of Workstation itself.  Oh well. 

Thanks for all your help the past few days, bluefirestorm.  You've spared me a lot of headache/heartache.

0 Kudos
DhairyaT
VMware Employee
VMware Employee

@DMancini_XDS 

Requesting you to reproduce the issue and share the full support bundle of the affected VM using below- 

Help->Support->Collect Support Data 

0 Kudos
DhairyaT
VMware Employee
VMware Employee

@DMancini_XDS Please follow the workaround mentioned in this kb-  https://ikb.vmware.com/s/article/89855 and let us know if it works for you.

0 Kudos