VMware
1 2 3 Previous Next 43 Replies Last post: Sep 13, 2009 12:40 PM by disruptor  

Here is patch for Workstation 6.5.1 on Linux 2.6.29 kernel posted: Mar 31, 2009 2:39 AM

Click to view Krellan's profile Enthusiast 82 posts since
Jun 16, 2008
Here is something I'm proud of: a complete patch for getting VMware Workstation 6.5.1 to correctly compile its kernel modules against the new Linux 2.6.29 kernel.

It's attached.

I read the other threads here:

http://communities.vmware.com/message/1210084

http://communities.vmware.com/message/1141306

There were also patches posted to these threads, but after looking at them, I was concerned. There seems to be a serious error, as cap_raise() has been blindly changed to cap_raised(), which is actually a different API that has a completely different effect (it's a query instead of a set). Linux 2.6.29 changed capabilities a lot, evidently, and broke a lot of old code. I had to put in wrapper functions to call the capabilities change, because the new kernel shuffled things around quite a bit.

Also, the Makefile itself was failing to find the Linux kernel headers in many situations. This was a bug in VMware's Makefile: it wasn't including the $(HEADER_DIR) in its check for various features, which was causing misdetection of a lot of things earlier (the poll_initwait() problem).

I made what I believe are the minimum possible changes in order to get it to compile and load cleanly against 2.6.29. It's also in a single, unified, patch, that covers all 5 of the known working VMware modules (ignore vmppuser, it's pretty much dead code, I don't know why VMware still ships it, since it's not used by vmware-modconfig in any way that I could find).

To use this patch, download it from the attachment, gunzip it, then go to your /usr/lib/vmware/modules/source directory. Untar all of the files. Apply the patch with "patch -p1". After the patch has applied, re-tar all the files. Then, run "vmware-modconfig --console --install-all" again, under Linux 2.6.29, and it should now work. Please let me know your results, if it doesn't work. I tested it on an x86_64 machine with VMware Workstation 6.5.1 and it worked, but anything else is untested.

Exercises left to the reader:

1) Make a script to help people install this patch easier. It's error-prone to have to untar and re-tar files by hand.

2) Add in the KERNEL_VERSION checks to the code. Right now, I changed it just for 2.6.29. It could be made portable to other kernel versions as well by isolating the 2.6.29 changes in their own #if blocks.

Hope this helps!

Josh
Attachments:
Click to view depping's profile Champion 2,997 posts since
Jan 17, 2005
Cool, thanks! I will write a blog article about this DIY patch this afternoon. Been looking for a fix for a while but never had the time to actually write one myself! Much appreciated!!

Duncan
VMware Communities User Moderator


Blogging: http://www.yellow-bricks.com
Twitter: http://www.twitter.com/depping

If you find this information useful, please award points for "correct" or "helpful".
Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006
The first patch worked perfectly on 32 bit 2.6.29 kernel, and the patch executes fine on 64 bit, no errors, but fails to build the module with the following errors on 64 bit:
::::
sux <username>
password
vmplayer
Logging to /tmp/vmware-h2/setup-6847.log
modinfo: could not find module vmmon
modinfo: could not find module vmnet
modinfo: could not find module vmblock
modinfo: could not find module vmci
modinfo: could not find module vsock
modinfo: could not find module vmmon
modinfo: could not find module vmnet
modinfo: could not find module vmblock
modinfo: could not find module vmci
modinfo: could not find module vsock
/usr/bin/vmplayer: line 31: 6847 Segmentation fault "$BINDIR"/vmware-modconfig --appname="VMware Player" --icon="vmware-player"
::::

This is confirmed by another person too. I haven't tried the new script though the patch and first method was integrated into svmi vm installer script and worked fine with 32 bit. Didn't have as many checks because I figure that vmware will release a fixed version quite soon.

Thanks for your work.

By the way, another person said that the first rough patch, the one you fixed, worked on 64 bit, or at least appears to work.
Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006

Sorry for not providing more details: Both system are Debian Sid, recently upgraded.

I used the svmi script to run the patches and the main fixes from the original thread's one liner. You'd not yet put up your new stuff so I didn't check that or see it until checking back on the thread, though I can try using the script, but if it what it does is automate the original one liner basically, that's what svmi does too, download patch, do the tar stuff, run the patch, do the tar stuff, then quit. It does not try to build the module however, I leave that for the user to do when they start the vmplayer gui.

The errors came from trying to build the module using the gui module builder, to see where it failed, since the gui doesn't open at all.

Patches run fine, I assume the other parts run fine too.

Should not be a ram issue, 2 gigabytes on both system, actually 1 gigabyte on one. I've run this on two 32 bit installs with 2.6.29 kernel, works fine. Not sure what else to tell you.

And yes, it's vmplayer, never tried workstation, so that's a difference.

A sux start of vmplayer on the working 32 bit machine just shows the various things starting as expected.

Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006

on 64 bit, returns: segmentation fault

Error 139

Nothing else, error is from $? return value.



Click to view dawoim's profile Lurker 2 posts since
Apr 1, 2009

I tried out the patch, and am having a problem. I'm on SuSE Factory (almost current), kernel 2.6.29-rc8-5-pae. The script applied correctly, the modules built and installed OK. When I run the Player, it tries to start up, then I get a pop-up message, "Error while powering on: Cannot find a valid peer process to connect to." Here's the running processes:

# ps -ef | grep vm
root 28123 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-bridge -s 6 -d /var/run/vmnet-bridge-0.pid -n 0
root 28639 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-dhcpd -s 6 -cf /etc/vmware/vmnet1/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet1/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet1.pid vmnet1
root 28644 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-netifup -s 6 -d /var/run/vmnet-netifup-vmnet1.pid /dev/vmnet1 vmnet1
root 28655 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-dhcpd -s 6 -cf /etc/vmware/vmnet8/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet8/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet8.pid vmnet8
root 28657 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-natd -s 6 -d /var/run/vmnet-natd-8.pid -m /var/run/vmnet-natd-8.mac -c /etc/vmware/vmnet8/nat/nat.conf
root 28668 1 0 17:32 ? 00:00:00 /usr/bin/vmnet-netifup -s 6 -d /var/run/vmnet-netifup-vmnet8.pid /dev/vmnet8 vmnet8
root 29583 1 0 17:32 pts/2 00:00:01 /usr/lib/vmware/bin/vmware-tray
root 29658 15858 8 17:40 pts/2 00:00:04 /usr/lib/vmware/bin/vmplayer
root 30435 29658 0 17:40 pts/2 00:00:00 /usr/lib/vmware/bin/vmplayer-daemon
root 30460 1 3 17:41 pts/2 00:00:00 /usr/lib/vmware/bin/vmware-unity-helper --daemon
# lsmod | grep vm
vmnet 39860 13
vmblock 13552 3
vmci 50004 0
vmmon 76972 0

What's missing?


Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006
I tried a few things, deleting /etc/vmware and /usr/lib/vmware after removing vmplayer, then running hte patch, which works fine, then running your install script, which ran fine, then trying to run the vmplayer module build command, vmware-modconfig --console --install-all which exits with error 139, segmenation fault.

running as normal user: vmware-modconfig --help works fine every time.

Since there's a small chance this is based on a hardware problem, and since another user got vmplayer running on 2.6.29 sid with the new patch and running your script, which I believe svmi duplicated in terms of the logic, then running hte module building command, I'm going to need to see if anyone else has issues.

I'll let you know what other people experience, for me it's uninstallable, sadly, on 2.5.1 vmplayer, but happily, the 32 bit install works just fine, old patch and new one both are fine.
Click to view dawoim's profile Lurker 2 posts since
Apr 1, 2009

OK, I see what is going on. I was trying to display to the XWindows server on my Mac (OS X 10.4.11). At the tail of the vmware log file is:

Apr 01 21:26:00.482: vmx| XINFO X fd is 133
Apr 01 21:26:00.483: vmx| XINFO depth 24 bpp 32 class 4
Apr 01 21:26:00.507: vmx| XINFO IO fatal error. Exiting ...

In other words, it had a beef with my Mac's X server. When I run the vmware on the native display, it works OK. I hope that gets fixed sometime, as the display on my Mac is much larger than the one on the PC...


Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006
Here's an update on Debian Sid 2.6.29 kernel 2.5.2 install for vmplayer:

Without running the patch:

vmon module build failed, I can't find the error output, but the module started building, no seg fault, but failed on vmon.

after running the patch, which had this hunk error, but otherwise seemed to go ok, the module built, I don't know if it's missing any features:
Patch error:
:::::::::::::::
Applying patch now...
patching file vmblock-only/Makefile
Hunk #1 succeeded at 122 (offset 9 lines).
patching file vmci-only/Makefile
Hunk #1 succeeded at 122 (offset 9 lines).
patching file vmmon-only/include/x86apic.h
Hunk #1 FAILED at 94.
1 out of 1 hunk FAILED -- saving rejects to file vmmon-only/include/x86apic.h.rej
patching file vmmon-only/linux/driver.c
patching file vmmon-only/linux/hostif.c
Hunk #1 succeeded at 3424 (offset 1 line).
Hunk #2 succeeded at 3538 (offset 1 line).
Hunk #3 succeeded at 3662 (offset 1 line).
patching file vmmon-only/Makefile
Hunk #1 succeeded at 122 (offset 9 lines).
patching file vmnet-only/Makefile
Hunk #1 succeeded at 122 (offset 9 lines).
patching file vmnet-only/netif.c
patching file vsock-only/Makefile
Hunk #1 succeeded at 122 (offset 9 lines).

::::::::::::::::

Now vmplayer starts fine with the patch on 64 bit. Any chance of getting a new patch for 2.5.2? The old one seems to work fine except for the bad hunk.
Click to view h2-2's profile Novice 15 posts since
Nov 19, 2006
It's the same in 32 and 64 bit, so here's the error without the patch for 2.5.2 on Debian Sid 2.6.29 kernel:

::::::::::
Stopping VMware services:
Virtual machine communication interface done
Virtual machine monitor done
Blocking file system done
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmmon-only'
make -C /lib/modules/2.6.29-0.lqx.1-liquorix-686/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.29-0.lqx.1-liquorix-686'
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/driver.o
In file included from /tmp/vmware-root/modules/vmmon-only/linux/driver.c:31:
/tmp/vmware-root/modules/vmmon-only/./include/compat_wait.h:78: error: conflicting types for 'poll_initwait'
include/linux/poll.h:67: error: previous declaration of 'poll_initwait' was here
/tmp/vmware-root/modules/vmmon-only/linux/driver.c: In function 'LinuxDriverSyncCallOnEachCPU':
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1423: error: too many arguments to function 'smp_call_function'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c: In function 'LinuxDriver_Ioctl':
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1987: error: 'struct task_struct' has no member named 'euid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1987: error: 'struct task_struct' has no member named 'uid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1988: error: 'struct task_struct' has no member named 'fsuid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1988: error: 'struct task_struct' has no member named 'uid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1989: error: 'struct task_struct' has no member named 'egid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1989: error: 'struct task_struct' has no member named 'gid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1990: error: 'struct task_struct' has no member named 'fsgid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1990: error: 'struct task_struct' has no member named 'gid'
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:2007: error: too many arguments to function 'smp_call_function'
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/driver.o] Error 1
make[1]: *** _module_/tmp/vmware-root/modules/vmmon-only Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.29-0.lqx.1-liquorix-686'
make: *** http://vmmon.ko Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmmon-only'
Unable to install vmmon
ERROR: 1 occurred in the Vmplayer kernel module build!
;:::::::::

I don't know how important that failed hunk was in the patch, but the module builds, and vmplayer starts fine now.

VMware Developer

SDKs, APIs, Videos, Learn and much more in the Developer community.

Learn More

Developer Sample Code

Increase your developer productivity with VMware API sample code.

Learn More

VMworld Sessions & Labs

Online access to the latest VMworld Sessions & Labs and online services.

Learn more

Purchase PSO Credits Online

Purchase credits to redeem training and consulting services online.

Buy Now

Community Hardware Software

View reported configurations or report your own.

Learn More

VMware vSphere

Come witness the next giant leap in virtualization.

Register Today

Communities