VMware Communities
BitOBear
Enthusiast
Enthusiast

Workstation Installer Hang 6.5.3 and Workaround

I just encountered a hang installing Workstation on Ubuntu Karmic.

Symptom: you are running a an install using 6.5.3 bundle and you get to "Configuring VMWare Player" and then everything stops indefinitely.

Cause: the installer runs vmware-modconfig-console which produces enough output with some kernels that the python engine deadlocks is command piplining during the compile.

Technical Details: It is always dangerous for a linux/unix program to use the pipe-based "run a sub-command" library calls if it isn't ready to read all the output in one thread while waiting for the command to complete in another. Normally such commands should either be reading the result text or redirecting that result text to /dev/null. The naieve version just runs the command and waits for the result. But if the result text is larger than the system pipe buffer size, the command will block while trying to send its output, and the command never finishes. With the command waiting for the parent to read the data, and the parent waiting for the command to finish, the whole shebang just comes to a stop.

Why this is happening: The current driver patches for the 2.6.31 kernels produce a lot of warning messages about the symbols MSVCVER and __FREE_BSD__ being not defined. This never happened before, and whoever wrote the installer never tried it against a long stream of output. So with the new warning stream the compile and the installer deadlock as above.

Workaround: There are three phases to the work around; It is necessary to catch and kill the compile step; this, in turn would cause the installer to back out the installation, so you have to stop that from happening; then you have to compile the modules manually.

Steps:

1) open two command windows.

2) become root in one of those windows. i.e. "su -". (If you havn't set your root password in Ubuntu use "sudo su -" and your login password)

3) in the root-ed window run "while true; do killall -9 vmware-modconfig-console; sleep 1; done" this wil try, once each second, to kill the module compile step. Ignore the stream of not-found messages.

4) in the non-root window run the install with sudo and the --ignore-errors argument; i.e. "sudo ./VMware-Workstation-6.5.3-185404.i386.bundle --ignore-errors". This will take a little longer than usual because of the continuous killall above, don't sweat it.

5) once the installer finishes go back to the root window and do a control-C to stop the kill loop.

6) In the root window run "vmware-modconfig --console --install-all" to install the modules.

Caveats and Tips:

a) Once you have a hung installer session you have to kill the python command that is running the installer. Think of that as a step-0 if it comes up. 8-)

b) If the installer still hangs, the kill loop may not be fast enough; start over but leave the 'sleep 1;' out of the kill loop. It will slow down the install even more, but it is more likely to catch the command in time.

ASIDE: It woudl be nice if the installer had a don't compile the modules step...

Hopefully the installer will be fixed soon.

Reply
0 Kudos
38 Replies
298
Contributor
Contributor

Thanks BitOBear, this worked like a charm. I'll buy you a beer.

Reply
0 Kudos
jcottier
Contributor
Contributor

I am running Ubuntu 8.04 and I tried all the fixes here and others (similar problem on FC11 apparently). But nothing works. I have tried had to go back to 6.5.2, but now that has the same error!

Any ideas would be most welcome!

Reply
0 Kudos
TNowakowski
Contributor
Contributor

I had the same problem and for me worked a little different solution.

I discovered that install process hangs while compiling (make) modules. For my Ubuntu 8.10 the only working way of pushing installer over 62% of progress was simply changing binary file 'make' for binary file 'true'.

To be clear. I've made a copy of original make program and replaced it by true program (yes 'true' is a small program which always returns true). After that switch I run the installer and everything went fine without my intervention. After the installer finished his hard job, I recovered my make program and now I got stuck with fully installed Workstation without modules. The only thing I have got to do now is to compile Workstation modules under my custom made kernel 2.6.30.

I will try to compile my modules under stock Ubuntu 8.10 (2.6.27-14) if I don't find a solution soon.

-- Think twice or even more - Vbrain
Reply
0 Kudos
jcottier
Contributor
Contributor

I have found that if I kill the GUI installer when it has hung (you need to monitor the end of /var/log/vmware-installer for a while to see when it has actually stopped) then run "vmware-modconfig --console --install-all" as root. That completes the install, but only gives a working VMPlayer, no Workstation. Still, at least it allows me to run the OS's. See also this thread http://communities.vmware.com/thread/227575?tstart=30

Presumably a kernel update has messed things up. I hope they fix it soon.

Reply
0 Kudos
Peter_S_
Contributor
Contributor

WONDERFUL!! Heart

worked great for Fedora 11, too.

Thank you!

Peter

Reply
0 Kudos
298
Contributor
Contributor

jcottier, I had the same problem at one time. I think the whole point of BitOBear's and also TNowakowski's idea is to force the VMware installer through the install procedure regardless of the failures in module compilation. When compilation is disabled either by killing it all the time (BitOBear) or by replacing make with a NoOp, (TNowakowski), then the installer will at least copy everything into the correct directories. In a second step, we can then compile the modules.

Reply
0 Kudos
guycook
Contributor
Contributor

Peter S, have you written this anywhere for installing Workstation 6.5.3 on a fresh Fedora 11 install? I would sure like to follow that, having issues with the install hanging on the VMware Player step of the install process.

Guy C

Reply
0 Kudos
Peter_S_
Contributor
Contributor

Hi Guy,

I had VMware Workstation 6.5.2 installed before with the tricks from this posting:

I uninstalled version 6.5.2 with rpm and reinstalled Version 6.5.3 with the .bundle installer with the tutorial from BitOBear above.

I tried the installation with the .rpm installer but it didn't work. So i used the .bundle and it works perfect.

Between uninstall and install i updated my Fedora (with restart).

Hope it helps

Peter

Reply
0 Kudos
guycook
Contributor
Contributor

Peter,

Thanks, I got 6.5.3 installed too, I did it by doing first rpm -e of the previous failed attempt install.

2. Opened a second terminal as root.

3. In the first terminal I rpm installed the package, stopping when prompted for the EULA, go to the second terminal and

4. type "mv -v /etc/vmware/database /tmp/ hit enter that will move the database to tmp

5. Go to the first terminal, accept the EULA and finish the install, when it says "Installation Successfull" go to the 2nd terminal and move the database back by typing "mv -v /tmp/database /etc/vmware/ to move the database back

6. Then the important part in the terminal as root "vmware-modconfig --console --install-all" you'll see lots of messages but ignore them and eventually it will finish, you can then start vmware, and install guest os.

Thanks to all who posted the information that built the recipe above.

Guy Cook

Reply
0 Kudos
jcottier
Contributor
Contributor

Thanks Peter, that procedure worked perfectly in Kubuntu 8.04 too.

Regards John Cottier.

Reply
0 Kudos
phoenix89
Contributor
Contributor

AWESOME, Thanks for your help.

Reply
0 Kudos
Number7
Contributor
Contributor

Brilliant.

Thank you Guy.

Reply
0 Kudos
znmeb
Contributor
Contributor

I ran into this on openSUSE 11.2 Milestone 8 but it seems to have gone away with the more recent 11.2 Release Candidate 1. I thought it was openSUSE-specific, so I filed a bug there.

Reply
0 Kudos
WalterKasberg
Contributor
Contributor

Hi,

this worked also perfectly with Mandriva 2010 rc2 and kernel 2.6.31.

Thanks

Walter

Reply
0 Kudos
dummyooo
Contributor
Contributor

Just perfect. Thanks for your effort!!!

Reply
0 Kudos
canuck75
Contributor
Contributor

I can't get it to work. The install does not hang up anymore but VMWare deconfigures itself and removes its software before exiting. What am I missing? I followed all the steps above.

Reply
0 Kudos
guycook
Contributor
Contributor

HI,

I just reinstalled, the steps work. List them here please so I can

review them in the morning Are you installing on Fedora 11 32bit?

Guy

Reply
0 Kudos
canuck75
Contributor
Contributor

I am using Ubuntu Karmic 64 bit.

1) I start a terminal in root and run the while loop code in here.

2) I start another terminal and install the VMWare package with sudo.

3) I follow the prompts from VMWare and the software installs.

4) After it gets done configuring (where it used to lock) it now continues but deconfigures and uninstalls the software. I get an error saying install was unsuccessful.

Reply
0 Kudos
guycook
Contributor
Contributor

Here's what I do.

Terminal 1 and 2 get opened as root.

#1 I start the install of the rpm, and when the install asks me to approve the license, I then start #2

#2 terminal I enter mv -v /etc/vmware/database /tmp/ that is done back to #1

#1 accept the license, and then let it install until it says "Installation Successful" then to #2 terminal

#2 terminal enter mv -v /tmp/database /etc/vmware/ to move the database back

#2 I then in the same #2 terminal type "vmware-modconfig --console --install-all

that's it, when the modconfig is all done your vmware install is done.

Reply
0 Kudos