Re: VMware Workstation 7.1.3 Linux kernel 2.6.37 (and 2.6.36) patch

VMware Communities

UPDATE: These patches are obsoleted by Workstatioin 7.1.4, for which only a BKL-less kernel needs any patch (to vmnet).

Attached is the patch for VMware Workstation 7.1.3 on Linux Kernel 2.6.37. Tested against Ubuntu Natty 11.04 2.6.37-2-generic.

The patch also includes the necessary change for 2.6.36, and should work with both kernels, since the 2.6.37 change is protected by a version check. The plain 2.6.36 patch, which does not support 2.6.37 is available here.

List of changes:

define VMW_HAVE_UNLOCKED_IOCTL in vmmon's driver.c as required for 2.6.36
the semaphore patch for 2.6.37, applied only to those modules that actually need it: vmci, vmnet, and vsock.

[UPDATE: The attached patch is updated to work through 2.6.37-11]

Updated to reflect Workstation 7.1.4.

109 Replies

Sorry for the noise. I had downloaded one of the 'other' patches in this thread, that had the vmci part missing..

Patched succesfully now. Thank you.

Tested with kernel 2.6.37-11.25 in Linux Mint 10 64 bits, all OK.

Is this patch compatible with kernel 2.6.37-12.26 ? (I think that is 2.6.37 final)

Hi,

i tried to use the 2 following patches from here for my kernel-2.6.37 with Workstation 7.1.3 on Fedora 14,

but this fails with errors.

http://communities.vmware.com/servlet/JiveServlet/download/1674972-52659/vmware-7.1.3-2.6.37.patch.z...

http://communities.vmware.com/servlet/JiveServlet/download/1674972-52660/patch-modules_2.6.37.sh

[root@fc14 tmp]# ./patch-modules_2.6.37.sh 
patching file vmmon-only/linux/driver.c
Stopping VMware services:
   VMware USB Arbitrator                                   [  OK  ]
   VM communication interface socket family                [  OK  ]
   Virtual machine communication interface                 [  OK  ]
   Virtual machine monitor                                 [  OK  ]
   Blocking file system                                    [  OK  ]
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmmon-only'
make -C /lib/modules/2.6.37-1.fc14.x86_64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/kernels/2.6.37-1.fc14.x86_64'
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/driver.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/iommu.o
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function 'IOMMU_SetupMMU':
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:156:7: error: implicit declaration of function 'iommu_map_range'
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function 'IOMMU_VMCleanup':
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:403:7: error: implicit declaration of function 'iommu_unmap_range'
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c: In function 'IOMMUUnregisterDeviceInt':
/tmp/vmware-root/modules/vmmon-only/linux/iommu.c:216:17: warning: ignoring return value of 'device_attach', declared with attribute warn_unused_result
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/iommu.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/2.6.37-1.fc14.x86_64'
make: *** [vmmon.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmmon-only'
Unable to install vmmon


All done, you can now run VMWare WorkStation.
Modules sources backup can be found in the '/usr/lib/vmware/modules/source-workstation7.1.3-2011-01-08-13:40:40-backup' directory

OK, maybe teeny progress here. On Debian Sid (aptosid) with this kernel:

2.6.37-0.slh.1-aptosid-amd64

with a clean new installation of Workstation 7.1.3, I did the following:

1. Changed the vmware-7.1.3-2.6.37-9-generic.patch file name to "vmware-7.1.3-2.6.37-generic.patch".

2. Changed the "fpatch" reference, line 5 in "patch-modules_v61.sh", to "vmware-7.1.3-2.6.37-generic.patch".

3. Commented out the UID check which continues to throw a "you're not root" error.

After doing these, I ran the patch and saw this:

root@aptosidbox:/tmp# sh patch-modules_v61.sh
patching file vmci-only/include/compat_semaphore.h
patching file vmmon-only/linux/driver.c
patching file vmnet-only/compat_semaphore.h
patching file vsock-only/shared/compat_semaphore.h
Stopping VMware services:
   VMware USB Arbitrator                                               done
   VM communication interface socket family                            done
   Virtual machine communication interface                             done
   Virtual machine monitor                                             done
   Blocking file system                                                done
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmmon-only'
make -C /lib/modules/2.6.37-0.slh.1-aptosid-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
          MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.37-0.slh.1-aptosid-amd64'
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/driver.o
/tmp/vmware-root/modules/vmmon-only/linux/driver.c: In function ‘LinuxDriver_Ioctl’:
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1736: error: implicit declaration of function ‘unlock_kernel’
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1742: error: implicit declaration of function ‘lock_kernel’
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.37-0.slh.1-aptosid-amd64'
make: *** [vmmon.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmmon-only'
Unable to install vmmon
-e

All done, you can now run VMWare WorkStation.
Modules sources backup can be found in the '/usr/lib/vmware/modules/source-7.1.3-2011-01-08-09:26:42-backup' directory
root@aptosidbox:/tmp#
So it didn't finish -- but it went farther than I've previously seen. Any hint on a next step?
Thanks!

UnderXP wrote:
Is this patch compatible with kernel 2.6.37-12.26 ? (I think that is 2.6.37 final)

The patch that I had updated at the top of the thread works fine for me with Ubuntu Natty 2.6.37-12.26.

TheBigRed wrote:

i tried to use the 2 following patches from here for my kernel-2.6.37 with Workstation 7.1.3 on Fedora 14,
but this fails with errors.

The patch to which you refer doesn't appear sufficient. Try the one I posted at the top of the thread, which works with the latest 2.6.37 as released by Ubuntu (and, reportedly, by others).

Personally, I haven't tried 2.6.37 on Fedora. I'm using the stock Fedora kernel as well as the CCRMA real-time kernel for Fedora 14, for which I have a separate patch.

Thanks Noel!!

Hi Noe,

attached is the patch that works for me with Fedora 14 and (http://kojipkgs.fedoraproject.org/packages/kernel/2.6.37/1.fc15/src/kernel-2.6.37-1.fc15.src.rpm)

Thanks

The first error output that I'm seeing on my Debian system (see #44 above) is

In function ‘LinuxDriver_Ioctl’:
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1736: error: implicit declaration of function ‘unlock_kernel’
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1742: error: implicit declaration of function ‘lock_kernel’
I'm reading that kernel 2.6.37 drop the "Big Lock" -- at least for Arch and Debian kernels. Don't know about Ubuntu and Fedora. Is this relevant to the error on my system -- is there any obvious workaround? Thank you.

The initial error output on my Debian system (see #44 above) is:

In function ‘LinuxDriver_Ioctl’:
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1736: error: implicit declaration of function ‘unlock_kernel’
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1742: error: implicit declaration of function ‘lock_kernel’

A little slashdot reading indicates that Debian and Arch (at least) 2.6.37 kernels have dropped the "Big Kernel Lock". I don't know about Ubuntu and Debian 2.6.37's. Is this the problem with the patch? Is there any workaround - is this only a patching problem or does it go deeper for running VMware? Thank you.

attached is the patch that works for me with Fedora 14 and (http://kojipkgs.fedoraproject.org/packages/kernel/2.6.37/1.fc15/src/kernel-2.6.37-1.fc15.src.rpm)

Comparing the current patch (top of thread) with this one, they appear to be essentially identical (except for an embedded date and order of files), as makes sense. So it seems that we've got 2.5.37 covered on the various distros.

Heh -- I don't know was going on with the forum software early today --- I guess we now see what happens when a dumb, but very persistent user gets no feedback on his attempt to make a post. :smileyblush:

I'm running my VM on 2.6.26 until I learn enough to make a go of it on 2.6.27. Thanks for all efforts to support VMware on Linux.

I'm running 2.6.37-0.slh.7-aptosid-686 and WS 7.1.3... but i'm getting the below error...

root@roamer:~# vmware-modconfig --console --install-all
Stopping VMware services:
   VMware USB Arbitrator                                               done
   VM communication interface socket family                            done
   Virtual machine communication interface                             done
   Virtual machine monitor                                             done
   Blocking file system                                                done
Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmmon-only'
make -C /lib/modules/2.6.37-0.slh.7-aptosid-686/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
      MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.37-0.slh.7-aptosid-686'
CC [M] /tmp/vmware-root/modules/vmmon-only/linux/driver.o
/tmp/vmware-root/modules/vmmon-only/linux/driver.c: In function ‘LinuxDriver_Ioctl’:
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1736: error: implicit declaration of function ‘unlock_kernel’
/tmp/vmware-root/modules/vmmon-only/linux/driver.c:1742: error: implicit declaration of function ‘lock_kernel’
make[2]: *** [/tmp/vmware-root/modules/vmmon-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmmon-only] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.37-0.slh.7-aptosid-686'
make: *** [vmmon.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmmon-only'
Unable to install vmmon
root@roamer:~# uname -r
2.6.37-0.slh.7-aptosid-686

Can someone lead me in the right direction to get this back up an running??

dibl,

I saw this on the aptosid forums..

http://aptosid.com/index.php?name=PNphpBB2&file=viewtopic&p=4094&sid=ff3385cb06d3d347fe4a517ac350987...

      frapelli wrote:

      devil wrote:
the reason is that BKL (Big Kernel Lock) is history with 2.6.37 (at least with our kernels)
virtualbox and kvm work.

                Does it mean no hope to run vmware again on aptosid even in short future?

That you must ask the vmware guys. We don't know, if they are able to remove BKL dependency in their kernel modules.

Btw, vmware doesn't support 2.6.37 at the moment, without inofficial patches.

Sooo seems like we may be outof luck 😞 .... Until vmware or aptosid makes some changes

Yes, I was aware of that thread. The nuances of kernel-patching are well over my head, to be honest. But, 2.6.36 (after patching) still runs VMware fine. And of course, there's that other virtualization platform .....

I assume, based on some years of experience, that we'll see a real VMware upgrade that fully addresses the kernel changes in 2.6.37. Until then, we wait and cope.

i can no longer get the headers for for my 2.6.36-2.slh.3-aptosid-686 😞

Install Apate and you get 2.6.36, including headers.

Just wanted to say thanks to everybody that worked on this solution.

After having downgraded my kernel to 2.6.35 to get workstation working on my previous laptop, I decided that it was the wrong way of doing things (downgrading one product to get another to work). Using this patch and the script that was made for it, I could get the product with the problem (workstation) to play nice with the kernel.

Gents, a job well done.

:smileylaugh:

I assume, based on some years of experience, that we'll see a real VMware upgrade that fully addresses the kernel changes in 2.6.37. Until then, we wait and cope.

I already posted the patch to handle it. If you can't get it working, odds are you have the wrong patch.

OK, well, it's certainly possible that I made a misstep somewhere. I used the patch attached to the first post on this thread, and ran it as described in post #44 above, with the error output as shown. Any suggestions for a different approach would be appreciated. OS and hardware details are:

Host/Kernel/OS "aptosidbox" running Linux 2.6.37-0.slh.7-aptosid-amd64 x86_64 [ aptosid 2010-03 Ἀπάτη - kde-lite - (201012262151) ]
CPU Info        4x Intel Core i7 950 @ 8192 KB cache flags( sse3 ht nx lm vmx ) clocked at [ 4216.393 MHz ]
Videocard       nVidia GF100 [GeForce GTX 480] X.Org 1.7.7 [ 1920x1200@50.0hz ]
Network cards   Marvell 88E8056 PCI-E Gigabit
Processes 194 | Uptime 17:28 | Memory 904.2/5979.1MB | HDD KINGSTON SS100S2,OCZ-REVODRIVE,OCZ-REVODRIVE,WDC WD1002FAEX-0,WDC WD1002FAEX-0 Size 2136GB (2%used) | GLX Renderer GeForce GTX 480/PCI/SSE2 | GLX Version 4.1.0 NVIDIA 260.19.29 | Client Shell | Infobash v3.34