VmWare player & kernel 2.6.37

[#1]

Hi all,

I'm trying to launch VmWare 3.1.3 in a ubuntu 10.10 unvironment.

I compiled and installed the latest kernel vanilla from www.kernel.org, I modified the vmmon, vsock etc.to be compiled, but at the last I receive this error:

Starting VMware services:
   VMware USB Arbitrator                                               done
   Virtual machine monitor                                            failed
   Virtual machine communication interface                            failed
   VM communication interface socket family                           failed
   Blocking file system                                               failed
   Virtual ethernet                                                   failed

ALL compiling end fine i.e. no errors, es.

make -C /lib/modules/2.6.37-custom/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
      MODULEBUILDDIR= modules
make[1]: ingresso nella directory "/usr/src/linux-2.6.37"
  CC [M]  /tmp/vmware-root/modules/vmci-only/linux/driver.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/linux/driverLog.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/linux/vmciKernelIf.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciContext.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciDatagram.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciDriver.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciDs.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciEvent.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciGroup.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciHashtable.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciProcess.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciQueuePair.o
  CC [M]  /tmp/vmware-root/modules/vmci-only/common/vmciResource.o
  LD [M]  /tmp/vmware-root/modules/vmci-only/vmci.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /tmp/vmware-root/modules/vmci-only/vmci.mod.o
  LD [M]  /tmp/vmware-root/modules/vmci-only/vmci.ko
make[1]: uscita dalla directory "/usr/src/linux-2.6.37"

what's wrong ??

When I start vmware a loto of windows tell me that cannot be loaded the /dev/vmmon is not loaded.. the virtual machine is busy...

I don't how to fix it.

Help me !! 🙂

PS thos os the patch applied:

diff -ru original//vmci-only/include/compat_semaphore.h patched//vmci-only/include/compat_semaphore.h
--- original//vmci-only/include/compat_semaphore.h    2010-11-11 15:37:25.000000000 -0500
+++ patched//vmci-only/include/compat_semaphore.h    2010-11-20 10:11:56.000000000 -0500
@@ -28,7 +28,7 @@
#endif


-#if defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)
+#if (defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)) || LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37)
    /*
     * The -rt patch series changes the name of semaphore/mutex initialization
     * routines (across the entire kernel).  Probably to identify locations that
@@ -41,7 +41,7 @@
       #define DECLARE_MUTEX(_m)  DEFINE_SEMAPHORE(_m)
    #endif
    #ifndef init_MUTEX
-      #define init_MUTEX(_m) semaphore_init(_m)
+      #define init_MUTEX(_m) sema_init(_m,1)
    #endif
#endif

diff -ru original//vmmon-only/linux/driver.c patched//vmmon-only/linux/driver.c
--- original//vmmon-only/linux/driver.c    2010-11-11 15:37:22.000000000 -0500
+++ patched//vmmon-only/linux/driver.c    2010-11-29 23:09:16.000000000 -0500
@@ -145,7 +145,7 @@
#endif
#if LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 0) && \
     (defined(CONFIG_SMP) || defined(CONFIG_PREEMPT))
-#  define kernel_locked_by_current() kernel_locked()
+#  define kernel_locked_by_current() (current->lock_depth >= 0)
#else
#  define kernel_locked_by_current() 0
#endif
@@ -170,6 +170,7 @@
static int LinuxDriver_Ioctl(struct inode *inode, struct file *filp,
                            u_int iocmd, unsigned long ioarg);
#if defined(HAVE_UNLOCKED_IOCTL) || defined(HAVE_COMPAT_IOCTL)
+#define VMW_HAVE_UNLOCKED_IOCTL
static long LinuxDriver_UnlockedIoctl(struct file *filp,
                            u_int iocmd, unsigned long ioarg);
#endif
diff -ru original//vmnet-only/compat_semaphore.h patched//vmnet-only/compat_semaphore.h
--- original//vmnet-only/compat_semaphore.h    2010-11-11 15:37:23.000000000 -0500
+++ patched//vmnet-only/compat_semaphore.h    2010-11-20 10:11:56.000000000 -0500
@@ -28,7 +28,7 @@
#endif


-#if defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)
+#if (defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)) || LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37)
    /*
     * The -rt patch series changes the name of semaphore/mutex initialization
     * routines (across the entire kernel).  Probably to identify locations that
@@ -41,7 +41,7 @@
       #define DECLARE_MUTEX(_m)  DEFINE_SEMAPHORE(_m)
    #endif
    #ifndef init_MUTEX
-      #define init_MUTEX(_m) semaphore_init(_m)
+      #define init_MUTEX(_m) sema_init(_m,1)
    #endif
#endif

diff -ru original//vsock-only/shared/compat_semaphore.h patched//vsock-only/shared/compat_semaphore.h
--- original//vsock-only/shared/compat_semaphore.h    2010-11-11 13:04:44.000000000 -0500
+++ patched//vsock-only/shared/compat_semaphore.h    2010-11-20 10:11:56.000000000 -0500
@@ -28,7 +28,7 @@
#endif


-#if defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)
+#if (defined CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)) || LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37)
    /*
     * The -rt patch series changes the name of semaphore/mutex initialization
     * routines (across the entire kernel).  Probably to identify locations that
@@ -41,7 +41,7 @@
       #define DECLARE_MUTEX(_m)  DEFINE_SEMAPHORE(_m)
    #endif
    #ifndef init_MUTEX
-      #define init_MUTEX(_m) semaphore_init(_m)
+      #define init_MUTEX(_m) sema_init(_m,1)
    #endif
#endif

cheers

Angelo

[teicors]

Analyzing the error I realized:

The code is "well" compiled:

Using 2.6.x kernel build system.
make: ingresso nella directory "/tmp/vmware-root/modules/vmmon-only"
make -C /lib/modules/2.6.37-custom/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
      MODULEBUILDDIR= modules
make[1]: ingresso nella directory "/usr/src/linux-2.6.37"
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/driver.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/driverLog.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/hostif.o
/tmp/vmware-root/modules/vmmon-only/linux/hostif.c: In function ‘HostIFReadUptimeWork’:
/tmp/vmware-root/modules/vmmon-only/linux/hostif.c:2004: warning: ‘newUpBase’ may be used uninitialized in this function
  CC [M]  /tmp/vmware-root/modules/vmmon-only/linux/iommu.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/comport.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/cpuid.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/hashFunc.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/memtrack.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/phystrack.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/task.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/common/vmx86.o
  CC [M]  /tmp/vmware-root/modules/vmmon-only/vmcore/moduleloop.o
  LD [M]  /tmp/vmware-root/modules/vmmon-only/vmmon.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /tmp/vmware-root/modules/vmmon-only/vmmon.mod.o
  LD [M]  /tmp/vmware-root/modules/vmmon-only/vmmon.ko
make[1]: uscita dalla directory "/usr/src/linux-2.6.37"
make -C $PWD SRCROOT=$PWD/. \
      MODULEBUILDDIR= postbuild
make[1]: ingresso nella directory "/tmp/vmware-root/modules/vmmon-only"
make[1]: "postbuild" è aggiornato.
make[1]: uscita dalla directory "/tmp/vmware-root/modules/vmmon-only"
cp -f vmmon.ko ./../vmmon.o
make: uscita dalla directory "/tmp/vmware-root/modules/vmmon-only"

but the service don't start:

Built vsock module
Starting VMware services:
   VMware USB Arbitrator                                               done
   Virtual machine monitor                                            failed
   Virtual machine communication interface                            failed
   VM communication interface socket family                           failed
   Blocking file system                                               failed
   Virtual ethernet                                                   failed
Unable to start services

because:

FATAL: Error inserting vmmon (/lib/modules/2.6.37-custom/misc/vmmon.ko): Invalid module format

I don't know what I have to do !

[teicors]

I had to install the latest kernel compiled, but not installed.... 🙂 Now it works fine.