Re: Can't install VMWare Tools on Debian Squeeze

In order to rebuild vmware tools you need to install build-essential and linux-headers-2.6.32-5-amd64. Installing the headers brings in gcc-4.3 as a dependency.

If the packages are already installed what does "gcc -v" show? what is the output of ls "/usr/bin/gcc*" There should be a gcc-4.3. You can change the gcc version your using with "export CC=/usr/bin/gcc-4.3".

jeff

Every package previously installed should be current, thanks to dist-upgrade.

Package build-essential is 11.5, ...

Ah, I see, the package name is different for each gcc version!

gcc -v shows: gcc version 4.4.5 (Debian 4.4.5-8)

>> You can change the gcc version your using with "export CC=/usr/bin/gcc-4.3".

Thanks; I'll try that.

I have a similar problem after updating Debian Sid to linux headers 2.6.37-1-686. I can install vmware tools but the shared folders etc option bales-out with the following:

/tmp/vmware-root/modules/vmhgfs-only/super.c:73: error: unknown field ‘clear_inode’ specified in initializer
make[4]: *** [/tmp/vmware-root/modules/vmhgfs-only/super.o] Error 1
make[3]: *** [_module_/tmp/vmware-root/modules/vmhgfs-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.37-1-686'
make: *** [vmhgfs.ko] Error 2

Just for clarity:

ggc -v gives pkgversion='Debian 4.4.5-11'
I also used export CC=/usr/bin/gcc-4.3 to try and fix

===================================================

[EDIT]

On a German web-site I found the following patch. Does anyone know if is appropriate?

Anmeldungsdatum: 18.02.2010
Beiträge: 8

Verfasst am: 29.11.2010, 16:20 Titel:

Hallo zusammen,

habe nun eine Lösung gefunden:

ab dem 2.3.36 Kernel heißt clear_inode() nun evict_indode()

Daher diese Fehlermeldung:

Code:

/tmp/vmware-root/modules/vmhgfs-only/super.c:73: error: unknown field ‘clear_inode’ specified in initializer

Lösung:

1. in /usr/lib/vmware-tools/modules/source die Datei vmhgfs.tar entpacken.
2. im entpackten Verzeichnis die Datei super.c öffnen
3. hier wie folgt ändern:

Code:

#ifndef VMW_USE_IGET_LOCKED
    .read_inode    = HgfsReadInode,
#endif
    .clear_inode   = HgfsClearInode,
    .put_super     = HgfsPutSuper,
    .statfs        = HgfsStatfs,
};

in

Code:

#ifndef VMW_USE_IGET_LOCKED
    .read_inode    = HgfsReadInode,
#endif
    .evict_inode   = HgfsClearInode,
    .put_super     = HgfsPutSuper,
    .statfs        = HgfsStatfs,
};

4. Datei Speichern und gesamtes Verzeichnis wieder als vmhgfs.tar packen
5. vmware-config-tools.pl ausführen
6. Fertig, Shared Folders funktioniert wieder Very Happy

Gruß
AnJo

Now I'm getting an error building vmci.

All the others build OK.

--John

make[1]: Entering directory `/usr/src/linux-headers-2.6.32-5-amd64'
CC [M] /tmp/vmware-config0/vmci-only/kernelStubsLinux.o
CC [M] /tmp/vmware-config0/vmci-only/vmciDatagram.o
CC [M] /tmp/vmware-config0/vmci-only/vmci_drv.o
CC [M] /tmp/vmware-config0/vmci-only/vmciEvent.o
CC [M] /tmp/vmware-config0/vmci-only/vmciGuestDs.o
CC [M] /tmp/vmware-config0/vmci-only/vmciGuestKernelIf.o
CC [M] /tmp/vmware-config0/vmci-only/vmciKernelIf.o
In file included from /tmp/vmware-config0/vmci-only/vmciKernelIf.c:48:
/tmp/vmware-config0/vmci-only/pgtbl.h: In function ‘PgtblVa2MPN’:
/tmp/vmware-config0/vmci-only/pgtbl.h:301: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/pgtbl.h: In function ‘PgtblVa2Page’:
/tmp/vmware-config0/vmci-only/pgtbl.h:373: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/vmciKernelIf.c: In function ‘VMCIHost_SignalCall’:
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:320: error: ‘TASK_NORMAL’ undeclared (first use in this function)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:320: error: (Each undeclared identifier is reported only once
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:320: error: for each function it appears in.)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c: In function ‘VMCIHost_WaitForCallLocked’:
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:362: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:362: error: ‘TASK_INTERRUPTIBLE’ undeclared (first use in this function)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:370: error: implicit declaration of function ‘schedule’
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:378: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:378: error: ‘TASK_RUNNING’ undeclared (first use in this function)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:382: error: implicit declaration of function ‘signal_pending’
/tmp/vmware-config0/vmci-only/vmciKernelIf.c: In function ‘VMCI_SignalEvent’:
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:674: error: ‘TASK_NORMAL’ undeclared (first use in this function)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c: In function ‘VMCI_WaitOnEvent’:
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:704: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:704: error: ‘TASK_INTERRUPTIBLE’ undeclared (first use in this function)
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:716: error: dereferencing pointer to incomplete type
/tmp/vmware-config0/vmci-only/vmciKernelIf.c:716: error: ‘TASK_RUNNING’ undeclared (first use in this function)
make[4]: *** [/tmp/vmware-config0/vmci-only/vmciKernelIf.o] Error 1
make[3]: *** [_module_/tmp/vmware-config0/vmci-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-5-amd64'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/vmware-config0/vmci-only'
Unable to build the vmci module.

The communication service is used in addition to the standard communication
between the guest and the host. The rest of the software provided by VMware
Tools is designed to work independently of this feature.
If you wish to have the VMCI feature, you can install the driver by running
vmware-config-tools.pl again after making sure that gcc, binutils, make and the
kernel sources for your running kernel are installed on your machine. These
packages are available on your distribution's installation CD.
[ Press Enter key to continue ]

Just for clarity, I can confirm that if I apply the 'German' patch then I can get Shared Folders in Debian Sid. However as reported by JDlugosz I am unable to get vmci to work, and it bales-out with a similar message:

/tmp/vmware-root/modules/vmci-only/vmci_drv.c:96: error: unknown field ‘ioctl’ specified in initializer
/tmp/vmware-root/modules/vmci-only/vmci_drv.c:96: warning: initialization from incompatible pointer type
/tmp/vmware-root/modules/vmci-only/vmci_drv.c: In function ‘vmci_init’:
/tmp/vmware-root/modules/vmci-only/vmci_drv.c:156: error: implicit declaration of function ‘init_MUTEX’
make[4]: *** [/tmp/vmware-root/modules/vmci-only/vmci_drv.o] Error 1
make[3]: *** [_module_/tmp/vmware-root/modules/vmci-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-2.6.37-1-686'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmci-only'

While I can follow patch instructions like a trained monkey, I do not feel capable of defining what looks like a complete class viz: # include <sys/ioctl.h>, neither do I feel capable of deciding if this is the correct entry point to pursue or if the entry point can just be commented out.

Can anyone help?

Message was edited by: EdP Darn. This post http://communities.vmware.com/thread/293321?tstart=0 looks like it ought to handle the vmci issues unfortunately when I try some of the scripts I get a message saying 'workstation is not installed' - obviously I misread the thread and the patch only applies to Linux hosts. It is frustrating that there is no 'sticky' for current Workstation Tools issues as this seems to be a common problem (and ongoing problem).

It's too bad that they can't simply make an apt for VMWare Tools. At least for the most common build of Debian/Ubuntu!

Or, improve their install script so it packages up the results and uploads to their website. Then, the first person (on the bleeding edge of the Kernel) who has to recompile from source will provide it for everyone who follows.

It's silly enough that it mentions using the CC variable and tells you the version the kernal was built with, but doesn't put the two together, itself.

--John

To be fair - Debian Sid/Squeeze are not currently supported, and it appears that Kernel changes may also be affecting the way vmware works. I guess someone is now looking at these issues.

It would however be nice to have the most recent patches in one place with forks to the appropriate hosts. I really should not have to Google for possibly unsupported fixes.

That's surprising. I realize that Sid would not be supported, as is the nature of leading edge. But the current stable release certainly ought to be! And given the loooong shake-down period of Debian, there is no excuse to wait for the final release before beginning to update your own stuff for it.

Solved using the following items

Create a file named

vi /etc/profile.d/vmware_compiler.sh

end add the following line

export CC=/usr/bin/gcc-4.3

save the file and reboot guest using init 6 command.

now you can compile vmware tools without problem.

Greet

Omnibus

Hello Adcconsulting,

are sure with solution which you suggested? I tried to compile vmware-tools in guest machine with gcc-4.3 and I was unsuccessful:( My post is here:

http://communities.vmware.com/thread/302711?tstart=0

I think, this is not problem of distribution, but it has a relationship with kernel. Kernel in Debian Squeeze is 2.6.32. Post about this problem is here:

http://communities.vmware.com/message/1698097#1698097

Hi,

under my bin path i have the following items

root@pluto:/usr/bin# ls -la gcc*
lrwxrwxrwx 1 root root 7 Feb 22 16:33 gcc -> gcc-4.4
-rwxr-xr-x 1 root root 239528 Sep 21 03:01 gcc-4.3
-rwxr-xr-x 1 root root 255536 Nov 15 03:04 gcc-4.4

my ENV now contains

TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=eb56630155c6a0e5bdf6b19b000002db-1298631886.350047-1655394349
SSH_TTY=/dev/pts/0
USER=root
MAIL=/var/mail/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/usr/bin
LANG=en_US.UTF-8
SHLVL=1
HOME=/root
LANGUAGE=en_US:en
LOGNAME=root
CC=/usr/bin/gcc-4.3
_=/usr/bin/env
OLDPWD=/root

!!! note that for security reason some information has been removed!

uname -a

Linux pluto 2.6.32-5-amd64 #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux

root@pluto:/usr/bin# cat /etc/debian_version

6.0

and here the log of my install

root@pluto:~/vmware-tools-distrib# ./vmware-install.pl
A previous installation of VMware software has been detected.

The previous installation was made by the tar installer (version 4).

Keeping the tar4 installer database format.

Uninstalling the tar installation of VMware Tools.

Stopping VMware Tools services in the virtual machine:
   Guest operating system daemon:                                      done
   Unmounting HGFS shares:                                             done
   Guest filesystem driver:                                            done
   Guest memory manager:                                               done

File /etc/initramfs-tools/modules is backed up to
/etc/initramfs-tools/modules.old.0.

File /etc/updatedb.conf is backed up to /etc/updatedb.conf.old.0.

update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
The removal of VMware Tools 3.5.0 build-317866 for Linux completed
successfully. Thank you for having tried this software.

Installing the content of the package. This may take from several minutes to
over an hour depending on the size of the package.

In which directory do you want to install the binary files?
[/usr/bin]
What is the directory that contains the init directories (rc0.d/ to rc6.d/)?
[/etc]
What is the directory that contains the init scripts?
[/etc/init.d]

In which directory do you want to install the daemon files?
[/usr/sbin]

In which directory do you want to install the library files?
[/usr/lib/vmware-tools]

In which directory do you want to install the documentation files?
[/usr/share/doc/vmware-tools]
The installation of VMware Tools 3.5.0 build-317866 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command: "/usr/bin/vmware-uninstall-tools.pl".

Before running VMware Tools for the first time, you need to configure it by
invoking the following command: "/usr/bin/vmware-config-tools.pl". Do you want
this program to invoke the command for you now? [yes]

Stopping VMware Tools services in the virtual machine:
   Guest operating system daemon:                                      done
   Unmounting HGFS shares:                                             done
   Guest filesystem driver:                                            done
   Guest memory manager:                                               done
Trying to find a suitable vmmemctl module for your running kernel.

None of the pre-built vmmemctl modules for VMware Tools is suitable for your
running kernel. Do you want this program to try to build the vmmemctl module
for your system (you need to have a C compiler installed on your system)?
[yes]

Using compiler "/usr/bin/gcc-4.3". Use environment variable CC to override.

What is the location of the directory of C header files that match your running
kernel? [/lib/modules/2.6.32-5-amd64/build/include]

Extracting the sources of the vmmemctl module.

Building the vmmemctl module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmmemctl-only'
make -C /lib/modules/2.6.32-5-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.32-5-amd64'
CC [M] /tmp/vmware-config0/vmmemctl-only/backdoorGcc64.o
CC [M] /tmp/vmware-config0/vmmemctl-only/os.o
CC [M] /tmp/vmware-config0/vmmemctl-only/vmballoon.o
LD [M] /tmp/vmware-config0/vmmemctl-only/vmmemctl.o
Building modules, stage 2.
MODPOST 1 modules
CC /tmp/vmware-config0/vmmemctl-only/vmmemctl.mod.o
LD [M] /tmp/vmware-config0/vmmemctl-only/vmmemctl.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-5-amd64'
cp -f vmmemctl.ko ./../vmmemctl.o
make: Leaving directory `/tmp/vmware-config0/vmmemctl-only'
The module loads perfectly in the running kernel.

Extracting the sources of the vmhgfs module.

Building the vmhgfs module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmhgfs-only'
make -C /lib/modules/2.6.32-5-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.32-5-amd64'
CC [M] /tmp/vmware-config0/vmhgfs-only/backdoor.o
CC [M] /tmp/vmware-config0/vmhgfs-only/backdoorGcc64.o
CC [M] /tmp/vmware-config0/vmhgfs-only/bdhandler.o
CC [M] /tmp/vmware-config0/vmhgfs-only/cpName.o
CC [M] /tmp/vmware-config0/vmhgfs-only/cpNameLinux.o
CC [M] /tmp/vmware-config0/vmhgfs-only/cpNameLite.o
CC [M] /tmp/vmware-config0/vmhgfs-only/dbllnklst.o
CC [M] /tmp/vmware-config0/vmhgfs-only/dentry.o
CC [M] /tmp/vmware-config0/vmhgfs-only/dir.o
CC [M] /tmp/vmware-config0/vmhgfs-only/eventManager.o
CC [M] /tmp/vmware-config0/vmhgfs-only/file.o
CC [M] /tmp/vmware-config0/vmhgfs-only/filesystem.o
CC [M] /tmp/vmware-config0/vmhgfs-only/fsutil.o
CC [M] /tmp/vmware-config0/vmhgfs-only/hgfsBd.o
CC [M] /tmp/vmware-config0/vmhgfs-only/hgfsUtil.o
CC [M] /tmp/vmware-config0/vmhgfs-only/inode.o
CC [M] /tmp/vmware-config0/vmhgfs-only/kernelStubsLinux.o
CC [M] /tmp/vmware-config0/vmhgfs-only/link.o
CC [M] /tmp/vmware-config0/vmhgfs-only/message.o
CC [M] /tmp/vmware-config0/vmhgfs-only/module.o
CC [M] /tmp/vmware-config0/vmhgfs-only/page.o
CC [M] /tmp/vmware-config0/vmhgfs-only/request.o
CC [M] /tmp/vmware-config0/vmhgfs-only/rpcin.o
CC [M] /tmp/vmware-config0/vmhgfs-only/rpcout.o
CC [M] /tmp/vmware-config0/vmhgfs-only/staticEscape.o
CC [M] /tmp/vmware-config0/vmhgfs-only/stubs.o
CC [M] /tmp/vmware-config0/vmhgfs-only/super.o
LD [M] /tmp/vmware-config0/vmhgfs-only/vmhgfs.o
Building modules, stage 2.
MODPOST 1 modules
CC /tmp/vmware-config0/vmhgfs-only/vmhgfs.mod.o
LD [M] /tmp/vmware-config0/vmhgfs-only/vmhgfs.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-5-amd64'
cp -f vmhgfs.ko ./../vmhgfs.o
make: Leaving directory `/tmp/vmware-config0/vmhgfs-only'
The module loads perfectly in the running kernel.

Extracting the sources of the vmxnet module.

Building the vmxnet module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmxnet-only'
make -C /lib/modules/2.6.32-5-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.32-5-amd64'
CC [M] /tmp/vmware-config0/vmxnet-only/vmxnet.o
Building modules, stage 2.
MODPOST 1 modules
CC /tmp/vmware-config0/vmxnet-only/vmxnet.mod.o
LD [M] /tmp/vmware-config0/vmxnet-only/vmxnet.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-5-amd64'
cp -f vmxnet.ko ./../vmxnet.o
make: Leaving directory `/tmp/vmware-config0/vmxnet-only'
The module loads perfectly in the running kernel.

update-initramfs: Generating /boot/initrd.img-2.6.32-5-amd64
Extracting the sources of the vmblock module.

Building the vmblock module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmblock-only'
make -C /lib/modules/2.6.32-5-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.32-5-amd64'
CC [M] /tmp/vmware-config0/vmblock-only/linux/block.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/control.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/dbllnklst.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/dentry.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/file.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/filesystem.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/inode.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/module.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/stubs.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/super.o
LD [M] /tmp/vmware-config0/vmblock-only/vmblock.o
Building modules, stage 2.
MODPOST 1 modules
CC /tmp/vmware-config0/vmblock-only/vmblock.mod.o
LD [M] /tmp/vmware-config0/vmblock-only/vmblock.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-5-amd64'
cp -f vmblock.ko ./../vmblock.o
make: Leaving directory `/tmp/vmware-config0/vmblock-only'
The module loads perfectly in the running kernel.

Detected X.org version 7.6.7.

No drivers for X.org version: 7.6.7.

Skipping X configuration because X drivers are not included.

Starting VMware Tools services in the virtual machine:
   Switching to guest configuration:                                   done
   Guest memory manager:                                               done
   DMA setup:                                                          done
   Guest operating system daemon:                                      done

The configuration of VMware Tools 3.5.0 build-317866 for Linux for this running
kernel completed successfully.

You must restart your X session before any mouse or graphics changes take
effect.

You can now run VMware Tools by invoking the following command:
"/usr/bin/vmware-toolbox" during an X server session.

If you wish to configure any experimental features, please run the following
command: "vmware-config-tools.pl --experimental".

Enjoy,

--the VMware team

Maybe my comments don't apply in the Squeeze section as it looks like it may be a separate Sid kernel issue, even though there were similarities with the earlier posts.

However I tried 'enabling vmci' and running vmware-config-tools.pl again. Don't do this at home kids -- this immediately generated a kernel panic with the 2.6.37-1 kernel.

Stopping VMware Tools services in the virtual machine:
   Guest operating system daemon:                                       done
   Virtual Printing daemon:                                                    done
   VMware User Agent (vmware-user):                                    done
   Blocking file system:                                                         done
   Unmounting HGFS shares:
Message from syslogd@Sid at Feb 25 10:58:13 ...
kernel:[ 299.204444] ------------[ cut here ]------------

Message from syslogd@Sid at Feb 25 10:58:13 ...
kernel:[ 299.204468] invalid opcode: 0000 [#1] SMP

Message from syslogd@Sid at Feb 25 10:58:13 ...
kernel:[ 299.204474] last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/0000:02:03.0/usb1/1-0:1.0/uevent

For some reason this was not a very clean recovery and it took a couple of reboots, and fears of having to do another reinstall of Tools before the guest stabilised.

My host platform is ESX 3.5 not vmware-workstation

Bye

Thank you for letting me know this issue and a hint.

I resolved this completely, and posted my blog (http://blog.breadncup.com/2011/02/26/vmware-tool-patch-for-linux-kernel-2-6-3738/). You can grab the patches and apply them for complete vmware tool.

Just one note: I am using VMware Fusion and Guest OS is Linux 2.6.38 with Ubuntu 10.10.

Thanks,

Daniel

Hello Daniel,

thanks for your advice, but it unfortunatelly doesnt work, problem occured at the step 6.

Here is output from terminal:

polo@vmsqueeze:~/tmp/vmware-tools-distrib/lib/modules/source$ cd vmci-only && git apply --ignore-whitespace ../0001-vmwaretool-vmci-Linux-2.6.37.patch && cd ..
../0001-vmwaretool-vmci-Linux-2.6.37.patch:42: trailing whitespace.
static long vmci_ioctl(struct file *file,
error: shared/compat_semaphore.h: No such file or directory

Hi,

1. Are you in as same as environment like me? That is, are you using VMware Fusion and trying to use Linux as Guest OS and checked the version?

2. After Step 5, you should be in ~/tmp/vmware-tools-distrib/lib/modules/source. That is,


 cd ~/tmp/vmware-tools-distrib/lib/modules/source

before get to step#6.

Or you may check manually to see if vmci-only/shared/compat_semaphore.h is existed.

Let me know if it helps.

Daniel

Hi Daniel,

to tell the truth, my conditions are not the same but they are very similar. My software:

Host: Debian Lenny // this doesnt any influence on the solution

Guest: Debian Squeeze, kernel 2.6.32 // it may have influence on the solution, but in my opinion is it very similar to Ubuntu 10.10 (most importatnt is kernel)

Hypervizor: VMware server 2 // yes, that is the main difference between our cases

As for steps...

Of course, I was in ~/tmp/vmware-tools-distrib/lib/modules/source in step 5 and of course, I looked to vmci-only/ and of course I didnt find the directory "shared". BUT I found the file "compat_semaphore.h" directly in directory vmci-only/ I apologize, that I didnt notice immediately:(

So what do you recommend? Should I change the path in your scripts - I think, delete the "shared" directory from path? For example:

--- a/compat_semaphore.h

instead of

--- a/shared/compat_semaphore.h

I tried delete shared directory from path, but the errors always occured:

root@vmsqueeze:/home/polo/tmp/vmware-tools-distrib/lib/modules/source# cd vmci-only && git apply --ignore-whitespace ../0001-vmwaretool-vmci-Linux-2.6.37.patch && cd ..
../0001-vmwaretool-vmci-Linux-2.6.37.patch:42: trailing whitespace.
static long vmci_ioctl(struct file *file,
error: patch failed: compat_semaphore.h:28
error: compat_semaphore.h: patch does not apply

Do you think, this row in 0001-vmwaretool-vmci-Linux-2.6.37.patch file can affect the result? I think LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37) because I use kernel 2.6.32. But in your script is || operator... In my opinion, it can not affect the result.

+#if defined (CONFIG_PREEMPT_RT && LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 31)) || LINUX_VERSION_CODE >= KERNEL_VERSION(2, 6, 37)

Thanks for advice, I am sure, that solution of that problem helps all, who want to use Debian Squeeze as the guest machine.

P.S. Now, I have tried to install Ubuntu 10.10 kernel 2,6.35 as a guest and the problem is exactly the same as with using Debian Squeeze.

Right, polo939.

I should have mentioned that my patches are intended for higher version of linux kernel, like 2.6.37/38 on my the first post. It's not the one for Debian Squeeze itself or something like that since the base kernel from Debian Squeeze is 2.6.32, so I don't feel it's good idea to apply my patches directly in the case.

I've replied to JDlugosz, but I confused him with EdP who had a problem with 2.6.37. The intention of my post was for his case, not the originator, JDlugosz. Sorry for confusing you.

Daniel

Hi Daniel,

thank you for your patches and the accompanying very clear instructions. Unfortunately there appears to be a subtle difference between the kernel you are using and the Sid linux-headers-2.6.37-1-686 that I have installed.

The vmci and hgfs patches went ok with only minor issues. (no blank space in the bash "cd ..") command, and one warning:

static long vmci_ioctl(struct file *file,
warning: 1 line adds whitespace errors.

Unfortunately the vsock patch just gave out with the message

fatal: corrupt patch at line 22

I could not see anything obvious in the mutex statements in the patch except that possibly

- #define init_MUTEX(_m) semaphore_init(_m)

+ #define init_MUTEX(_m) sema_init(_m,1)

could be

- #define init_MUTEX(_m) semaphore_init(_m)

+ #define init_MUTEX(_m) semaphore_init(_m,1)

I did try this, but it also bombed out but maybe this is due to my kernel not being 'clean' at that stage.

Just for completeness there are the following statements in compat_semaphore.h in my install.

* The -rt patchset added the rename between 2.6.29-rt and 2.6.31-rt.
    */
   #ifndef DECLARE_MUTEX
      #define DECLARE_MUTEX(_m) DEFINE_SEMAPHORE(_m)
   #endif
   #ifndef init_MUTEX
      #define init_MUTEX(_m) semaphore_init(_m)
   #endif
#endif