opensolaris 2008.05 official release and vmware tools black screen when X starts

[ibodog2]

OpenSolaris 2008.05 installs fine on my VMware Workstation 6.0.3 running host on Windows XP Pro 32-bit. However, I encounter a problem I have not been able to solve with VMware tools. I create the /usr/dt/config/Xsession.d directory manually before running vmware-config-tools. Then when asked to create a new /etc/X11/xorg.conf file I select "no" (or "yes"). VMware tools then prints the messages that everything has installed fine. When I reboot the virtual machine I see the console start up messages. But when the X login manager should start up I only get a black, blank screen. I can ssh to the virtual machine, but have not been able to fix this problem from the command line. Luckily I have a snapshot taken just after the OS install to go back to. After much Googling and forum searching I have not yet found a solution to this problem. Can anyone here offer help?

[Asam]

I have the same problem with VMware Fusion 2.0b1 (89933) running on an early 2006 Intel iMac (2GHz Core Duo/2GB RAM/Mac OS X 10.5.2 - 9C7010).

When running OpenSolaris 2008.5 as guest OS following problems:

1. VMWare tools does not run when invoked from finder menu, though it does mount the VMware tools in the OpenSolaris desktop

2. Running the vmware-install.pl script as su then results in an error that /usr/dt/config/Xsession.d/9999.autostart-vmware-user.sh does not exist, and Execution aborted

3. After you manually make the dir under su, then copy the autostart-vmware-user.sh and re-run the vmware-install.pl script it runs as expecte, but then when you logout and reboot the guest OS, screen hangs after GRUB loader and you get black screen of no X-window death (BSNXWD)

There is a problem documented that causes VMware to load OpenSolaris guest in 64-bit if it detects 64-bit capable processor (though it shouldn't in my case as it's Core Duo),

http://blogs.sun.com/souvik/entry/workaround_for_vmware_fusion_hanging

But I tried it anyway, by adding "monitor_control.disable_longmode = 1" to my .vmx file in Mac OS X, that gets me out of the BSNXWD, and lands me in the System Maintenance Mode for OpenSolaris where it tells me

WARNING: The following files in / differ from the boot archive:

changed /etc/driver aliases

I can then try to clear the service and reboot in "Solaris failsafe" but it does the same thing and gets in some loop

4. Changing the .vmx file again on Mac side to remove "monitor_control.disable_longmode = 1" then just results in BSNXWD again

Only way around it I found was to reinstall VMware again (rolling back my TimeDrive) back to before VMware tools touched the system.

Have posted a few pictures of above in various stages, hope it's helpful to someone to avoid this problem and for VMware team to fix. VMware Fusion beta 2 looks great so far and once some of these problems are sorted should be a great product.

[Asam]

Perhaps I should have posted this to forum section for VMware Fusion Beta 2?

I'll link it back to here....

[sethmeisterg]

You can recover by replacing the vmware_drv.so (The Xorg video driver) with the one from before the tools installation (the following instructions assume a 32-bit guest):

\# cd /usr/X11/lib/modules/drivers

\# mv vmware_drv.so vmware_drv.so.new_vmware_tools

\# cp vmware_drv.so.BeforeVMwareToolsInstall vmware_drv.so

That resulted in my video working again.

[george_nussbaum]

Thanks for the tip sethemeisterg!!! Works for me!!