Contributor
Contributor

VMwareTools-8.8.4-730257 on Linux 3.6.1-1.fc17.x86_64 compile errors

Recently upgraded my Linux kernel to 3.6.1-1 and running into compile errors building vmhgfs for shared folders.  Ideas?

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmhgfs-only'
make -C /lib/modules/3.6.1-1.fc17.x86_64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/kernels/3.6.1-1.fc17.x86_64'
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/dir.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/message.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/hgfsUtil.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/request.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/link.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/cpName.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/rpcout.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/stubs.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/file.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/module.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/bdhandler.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/super.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/transport.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/tcp.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/vmci.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/fsutil.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/hgfsBd.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/cpNameLinux.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/page.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/filesystem.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/hgfsEscape.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/backdoor.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/inode.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/dentry.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/kernelStubsLinux.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/backdoorGcc64.o
  CC [M]  /tmp/vmware-root/modules/vmhgfs-only/cpNameLite.o
/tmp/vmware-root/modules/vmhgfs-only/dentry.c:43:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/dentry.c:43:4: warning: (near initialization for ‘HgfsDentryOperations.d_revalidate’) [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/file.c:128:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/file.c:128:4: warning: (near initialization for ‘HgfsFileFileOperations.fsync’) [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/page.c: In function ‘HgfsDoWriteBegin’:
/tmp/vmware-root/modules/vmhgfs-only/page.c:896:39: error: ‘KM_USER0’ undeclared (first use in this function)
/tmp/vmware-root/modules/vmhgfs-only/page.c:896:39: note: each undeclared identifier is reported only once for each function it appears in
/tmp/vmware-root/modules/vmhgfs-only/page.c:896:7: error: too many arguments to function ‘kmap_atomic’
In file included from include/linux/pagemap.h:10:0,
                 from /tmp/vmware-root/modules/vmhgfs-only/page.c:28:
include/linux/highmem.h:66:21: note: declared here
/tmp/vmware-root/modules/vmhgfs-only/page.c:904:36: error: macro "kunmap_atomic" passed 2 arguments, but takes just 1
/tmp/vmware-root/modules/vmhgfs-only/page.c:904:7: error: ‘kunmap_atomic’ undeclared (first use in this function)
/tmp/vmware-root/modules/vmhgfs-only/inode.c:121:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:121:4: warning: (near initialization for ‘HgfsDirInodeOperations.create’) [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:124:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:124:4: warning: (near initialization for ‘HgfsDirInodeOperations.mkdir’) [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:126:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:126:4: warning: (near initialization for ‘HgfsDirInodeOperations.lookup’) [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c: In function ‘HgfsPermission’:
/tmp/vmware-root/modules/vmhgfs-only/inode.c:1820:7: error: ‘struct hlist_head’ has no member named ‘next’
/tmp/vmware-root/modules/vmhgfs-only/inode.c:1820:7: warning: comparison of distinct pointer types lacks a cast [enabled by default]
/tmp/vmware-root/modules/vmhgfs-only/inode.c:1821:19: warning: initialization from incompatible pointer type [enabled by default]
make[2]: *** [/tmp/vmware-root/modules/vmhgfs-only/page.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [/tmp/vmware-root/modules/vmhgfs-only/inode.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmhgfs-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.6.1-1.fc17.x86_64'
make: *** [vmhgfs.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmhgfs-only'
0 Kudos
16 Replies
Contributor
Contributor

This should do the trick.

0 Kudos
Contributor
Contributor

Running into a patch failure, looks like in inode.c.  What is best way to resolve this?  Thanks.

[briano@bohalloran-dt ~]$ tar -zxf VMwareTools-8.8.4-730257.tar.gz
[briano@bohalloran-dt ~]$ cd vmware-tools-distrib
Resolving communities.vmware.com... 208.59.215.8, 208.59.215.10
Connecting to communities.vmware.com|208.59.215.8|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12946 (13K) [application/x-bash]
Saving to: `vmware_90_linux-3.6.x_patcher.sh'
100%[==================================================================================================>] 12,946      --.-K/s   in 0.001s 
2012-10-15 07:12:44 (16.0 MB/s) - `vmware_90_linux-3.6.x_patcher.sh' saved [12946/12946]
[briano@bohalloran-dt vmware-tools-distrib]$ chmod 755 vmware_90_linux-3.6.x_patcher.sh
[briano@bohalloran-dt vmware-tools-distrib]$ sudo ./vmware_90_linux-3.6.x_patcher.sh
------------------------ analysing module: vmblock ------------------------
patch for "vmblock" already applied (ignoring)
------------------------ cleaning up /tmp/tmp.ILyLHDCt4E ------------------------
------------------------ analysing module: vmhgfs ------------------------
------------------------ applying patch for vmhgfs ------------------------
patching file vmhgfs-only/dentry.c
patching file vmhgfs-only/inode.c
Hunk #1 FAILED at 75.
Hunk #2 FAILED at 952.
2 out of 5 hunks FAILED -- saving rejects to file vmhgfs-only/inode.c.rej
patching file vmhgfs-only/page.c
ERROR: Patch for "vmhgfs" failed.  Aborting.
------------------------ cleaning up /tmp/tmp.w9FdT1a4n4 ------------------------
[briano@bohalloran-dt vmware-tools-distrib]mce_markernbsp;
0 Kudos
Contributor
Contributor

Oops.  Solved the wrong problem for you there, bohalloran!  Which VMWare product are you using, exactly?  I can probably extend my script to support your version.

0 Kudos
Contributor
Contributor

Host OS is Mac OS X 10.6.8, running VMWare Fusion Version 4.1.3 (730298) with VMwareTools-8.8.4-730257.  My guest OS is

[briano@bohalloran-dt ~]$ uname -a
Linux bohalloran-dt 3.6.1-1.fc17.x86_64 #1 SMP Wed Oct 10 12:13:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
[briano@bohalloran-dt ~]
Thanks.
0 Kudos
Contributor
Contributor

I'm unable to test, as I don't have fusion, but give this a try.  Maybe your tools have already been patched somehow; these are designed to apply to a clean install of the guest tools, so you may need to uninstall them and reinstall a clean copy before these will apply.

Should be safe to try, so run it as before, and if it fails, then do a clean install and try again.

GL!

0 Kudos
Contributor
Contributor

All set!  Thanks for your help!

As you suggested, the key to getting this to work is to first run 'sudo ./bin/vmware-uninstall-tools.pl'

[briano@bohalloran-dt vmware-tools-distrib]$ sudo ./vmware_884-90_linux-3.6.x_patcher.sh

[sudo] password for briano:

------------------------ Testing patch "vmblock-90-36" ------------------------

patching file vmblock-only/linux/dentry.c

patching file vmblock-only/linux/inode.c

------------------------ patch "vmblock-90-36" applied cleanly. ------------------------

------------------------ committing patched sources for module vmblock ------------------------

------------------------ cleaning up /tmp/tmp.m3rYnVJwKJ ------------------------

------------------------ Testing patch "vmhgfs-884-36" ------------------------

patching file vmhgfs-only/dentry.c

patching file vmhgfs-only/file.c

patching file vmhgfs-only/filesystem.c

patching file vmhgfs-only/inode.c

patching file vmhgfs-only/Makefile.kernel

patching file vmhgfs-only/page.c

patching file vmhgfs-only/shared/autoconf/file_operations_fsync.c

patching file vmhgfs-only/shared/compat_fs.h

------------------------ patch "vmhgfs-884-36" applied cleanly. ------------------------

------------------------ committing patched sources for module vmhgfs ------------------------

------------------------ cleaning up /tmp/tmp.EAl5QfXi7Z ------------------------

------------------------ Testing patch "vmsync-90-36" ------------------------

patching file vmsync-only/sync.c

------------------------ patch "vmsync-90-36" applied cleanly. ------------------------

------------------------ committing patched sources for module vmsync ------------------------

------------------------ cleaning up /tmp/tmp.fL2gvbbRlB ------------------------

Success!  Now run vmware-config-tools.pl again, hopefully things go better.

[briano@bohalloran-dt vmware-tools-distrib]$

0 Kudos
Contributor
Contributor

I am trying to use this patch with VMwareTools-8.8.4-743747 (VMware Workstation 8.0.4 build-744019, Virtual Guest kernel 3.6.3) and the vmhgfs patch is failing:

------------------------ Testing patch "vmhgfs-884-36" ------------------------
patching file vmhgfs-only/dentry.c
patching file vmhgfs-only/file.c
Hunk #2 succeeded at 992 with fuzz 2.
patching file vmhgfs-only/filesystem.c
patching file vmhgfs-only/inode.c
Hunk #1 FAILED at 61.
Hunk #4 FAILED at 959.
2 out of 8 hunks FAILED
patching file vmhgfs-only/Makefile.kernel
patching file vmhgfs-only/page.c
patching file vmhgfs-only/shared/autoconf/file_operations_fsync.c
patching file vmhgfs-only/shared/compat_fs.h
------------------------ patch "vmhgfs-884-36" not applicable. ------------------------
------------------------ Testing patch "vmhgfs-90-36" ------------------------
patching file vmhgfs-only/dentry.c
patching file vmhgfs-only/inode.c
Hunk #1 FAILED at 75.
Hunk #2 FAILED at 952.
2 out of 5 hunks FAILED
patching file vmhgfs-only/page.c
------------------------ patch "vmhgfs-90-36" not applicable. ------------------------
ERROR: Could not find any working patch for module "vmhgfs".  Aborting.
------------------------ cleaning up /tmp/tmp.pD6Aa2idFT ------------------------

Any ideas? Interestingly enough, the vmsync patch never shows up in the script output. I dunno if it's just not getting to it or what.

Thanks for the work on the patch

thanks,

w

0 Kudos
Contributor
Contributor

I am in a similiar predicament with a slightly later version of the linux kernel. The host OS is Mac 10.7.5 and VMware fusion 4.1.3 (730298) with VMwareTools-8.8.4-743747.

$ uname -a
Linux fedora16 3.6.5-2.fc16.x86_64 #1 SMP Thu Nov 1 03:49:27 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

The later patch in this thread has supplied has been applied. This allows most of the tools to be installed except the vmxnet. which fails with the following errors:

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-root/modules/vmxnet-only'
make -C /lib/modules/3.6.5-2.fc16.x86_64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
  MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/kernels/3.6.5-2.fc16.x86_64'
  CC [M]  /tmp/vmware-root/modules/vmxnet-only/vmxnet.o
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:525:4: error: unknown field ‘get_rx_csum’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:525:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:525:4: warning: (near initialization for ‘vmxnet_ethtool_ops.get_eeprom_len’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:526:4: error: unknown field ‘set_rx_csum’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:526:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:526:4: warning: (near initialization for ‘vmxnet_ethtool_ops.get_eeprom’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:527:4: error: unknown field ‘get_tx_csum’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:527:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:527:4: warning: (near initialization for ‘vmxnet_ethtool_ops.set_eeprom’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:528:4: error: unknown field ‘set_tx_csum’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:528:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:528:4: warning: (near initialization for ‘vmxnet_ethtool_ops.get_coalesce’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:529:4: error: unknown field ‘get_sg’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:529:27: error: ‘ethtool_op_get_sg’ undeclared here (not in a function)
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:530:4: error: unknown field ‘set_sg’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:530:27: error: ‘ethtool_op_set_sg’ undeclared here (not in a function)
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:532:4: error: unknown field ‘get_tso’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:532:27: error: ‘ethtool_op_get_tso’ undeclared here (not in a function)
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:533:4: error: unknown field ‘set_tso’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:533:4: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:533:4: warning: (near initialization for ‘vmxnet_ethtool_ops.get_pauseparam’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:535:4: error: unknown field ‘get_ufo’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:535:27: error: ‘ethtool_op_get_ufo’ undeclared here (not in a function)
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c: In function ‘vmxnet_probe_device’:
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:1008:7: error: unknown field ‘ndo_set_multicast_list’ specified in initializer
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:1008:7: warning: initialization from incompatible pointer type [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:1008:7: warning: (near initialization for ‘vmxnet_netdev_ops.ndo_vlan_rx_add_vid’) [enabled by default]
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c: In function ‘vmxnet_map_pkt’:
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:2085:32: error: incompatible type for argument 2 of ‘pci_map_page’
include/asm-generic/pci-dma-compat.h:43:1: note: expected ‘struct page *’ but argument is of type ‘struct <anonymous>’
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:2104:26: error: incompatible type for argument 2 of ‘pci_map_page’
include/asm-generic/pci-dma-compat.h:43:1: note: expected ‘struct page *’ but argument is of type ‘struct <anonymous>’
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c: In function ‘vmxnet_rx_frags’:
/tmp/vmware-root/modules/vmxnet-only/vmxnet.c:2599:48: error: incompatible types when assigning to type ‘struct <anonymous>’ from type ‘struct page *’
make[2]: *** [/tmp/vmware-root/modules/vmxnet-only/vmxnet.o] Error 1
make[1]: *** [_module_/tmp/vmware-root/modules/vmxnet-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.6.5-2.fc16.x86_64'
make: *** [vmxnet.ko] Error 2
make: Leaving directory `/tmp/vmware-root/modules/vmxnet-only'

If I edit the source, and remove the unknown fields from the ethtool_ops structure, the vmxnet compiles but does not work. Specifically the Vmware vm has not netwok connections to the outside world.

0 Kudos
Contributor
Contributor

Here is a new patch for Linux 3.7.10.

0 Kudos
Contributor
Contributor

I hate to sound like a total idiot, but how exactly do I use the 130307_vmhgfs file?

I am trying to install Vmware tools on Kali Linux, the new replacement for Back Track, which haas a kernel of 3.7-trunk-amd64. I can't even get the patcher.sh to run, since it can't find /lib/vmware-tools/modules/source but I am assuming that is a totally different issue at this point.

I am running Vmware Workstation 9.0.2, and am not sure how to determine which version of tools is being used. I'll keep looking into that, but I would like info on how to use the last file you posted.

0 Kudos
Contributor
Contributor

I am running into the same problem on kali linux 1.0.1 x64 (debian wheezy).   I am running VMware Workstation 9.0.2 with VMwareTools-9.2.3-1031360.tar.gz

The process I am trying is:


echo cups enabled >> /usr/sbin/update-rc.d
echo vmware-tools enabled >> /usr/sbin/update-rc.d
apt-get install gcc make linux-headers-$(uname -r)
ln -s /usr/src/linux-headers-$(uname -r)/include/generated/uapi/linux/version.h /usr/src/linux-headers-$(uname -r)/include/linux/
ln -s /usr/src/linux-headers-$(uname -r)/include/generated/utsrelease.h /usr/src/linux-headers-$(uname -r)/include/linux/
ln -s /usr/src/linux-headers-$(uname -r)/include/generated/autoconf.h /usr/src/linux-headers-$(uname -r)/include/linux/
When the script tries starting the tools I am getting the following error:
/etc/init.d/vmware-tools: 1090: local: ': bad variable name
     Blocking file system:                                              failed
/etc/init.d/vmware-tools: 1187: local: ': bad variable name
   Guest operating system daemon:                                      done
Unable to start services for VMware Tools

When I look at both line 1090 and 1187 the code is:

local run_kver=`get_version_integer`

It appears the variable is not getting set for some reason or is out of range.

Any sugguestions would be appreciated

GP

0 Kudos
Contributor
Contributor

First off unpack module vmgf in vmware-tools-distrib/lib/modules/source and then copy the patch to it. Run patch -p0 --dry-run < [filename] to test the patch. To patch run patch -p0 < [filename]. Then archive module again and start installation. I use vmware 8 and the patch is only for vmgf.

0 Kudos
Contributor
Contributor

Hi,

I tried to apply the patch (130307_vmhgfs) for (VMwareTools-9.2.3-1031360 and Kali guest OS, kernel 3.7) and it raises several errors:

patching file vmhgfs-only/dentry.c
Hunk #1 FAILED at 36.
Hunk #2 FAILED at 71.
Hunk #3 FAILED at 80.
3 out of 3 hunks FAILED -- saving rejects to file vmhgfs-only/dentry.c.rej
patching file vmhgfs-only/inode.c
Hunk #1 FAILED at 61.
Hunk #2 FAILED at 74.
Hunk #3 FAILED at 729.
Hunk #4 FAILED at 951.
Hunk #5 FAILED at 1025.
Hunk #6 FAILED at 1107.
Hunk #7 succeeded at 1852 (offset 51 lines).
Hunk #8 FAILED at 1821.
7 out of 8 hunks FAILED -- saving rejects to file vmhgfs-only/inode.c.rej
patching file vmhgfs-only/Makefile.kernel
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file vmhgfs-only/Makefile.kernel.rej
patching file vmhgfs-only/page.c
Hunk #1 FAILED at 893.
Hunk #2 FAILED at 901.
2 out of 2 hunks FAILED -- saving rejects to file vmhgfs-only/page.c.rej
The next patch would create the file vmghfs.patched/shared/autoconf/file_operations_fsync.c,
which already exists!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored
patching file vmhgfs-only/shared/compat_fs.h
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file vmhgfs-only/shared/compat_fs.h.rej

Have you got more information?

Thanks,

D.

0 Kudos
Contributor
Contributor

I think it's because I wrote the patch for version 8? Can you install version 8 tools in workstation 9 and try the patch again? Maybe the hgfs driver can work with version 8? I'm not sure what this version 8 and version 9 is?

0 Kudos
Contributor
Contributor

Is it possible to download just the vmware tools (and the specific version you mention: 8)? Because I don't find them. I am just usign VMware Player 5.0.2.

Thanks for your prompt answer.

0 Kudos
Contributor
Contributor

It's not an answer but can you try the http://sourceforge.net/projects/open-vm-tools/files/open-vm-tools/ from version 8? I didn't try myself the version 9 but I think it's the same tools. Just report here when you cannot patch version 8?

Or you can get them here: http://packages.vmware.com/tools/index.html.

0 Kudos