ndavie
Contributor
Contributor

VM Workstation 16.1->16.2.1 on Ubuntu 21.10 broke everything

Ubuntu 21.10

I was annoyed to find shared VMs gone in 16, I was able to limp along and everything was fine until I "upgraded" and now I can't even create a VM.  

VMware Workstation unrecoverable error: (mks)
ISBRendererComm: Lost connection to mksSandbox (2878)
A log file is available in "/home/ndavie/Documents/vmware/AME 21H1/vmware.log".
You can request support.

To collect data to submit to VMware support, choose "Collect Support Data" from the Help menu.
You can also run the "vm-support" script in the Workstation folder directly.
We will respond on the basis of your support entitlement.

 

 

Labels (2)
59 Replies
tomarmistead
Contributor
Contributor

I'm in the same boat... After the upgrade, VMWare workstation build for the new kernel fails.

Tags (1)
shibu_nyc
Enthusiast
Enthusiast

I'm in the same boat with popOS 21.10... After the upgrade to 21.10, VMWare workstation build for the new kernel fails.

mlizbeth
Contributor
Contributor

You will need to clone this git branch and build vmnet and vmmon yourself.

https://github.com/mkubecek/vmware-host-modules/tree/workstation-16.2.1

Make sure you have the kernel headers for your version, gcc, build-essential, and any other build tools you might need.

tomarmistead
Contributor
Contributor

I'm getting errors when building this.
I'm on Pop OS 21.10;

This error shows up several times when compiling 
/home/tom/Downloads/vmware-host-modules/vmmon-only/./include/vm_asm_x86.h:67:44: error: expression in static assertion is not constant
67 | ASSERT_ON_COMPILE(sizeof(Selector) == 2 && \
| ~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
68 | ((__builtin_constant_p(expr) && ((expr) >> 16) == 0) || \
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
69 | sizeof(expr) <= 2))
| ~~~~~~~~~~~~~~~~~~

Ending in this;

make[3]: *** [scripts/Makefile.build:277: /home/tom/Downloads/vmware-host-modules/vmmon-only/common/task.o] Error 1
make[2]: *** [Makefile:1874: /home/tom/Downloads/vmware-host-modules/vmmon-only] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-5.15.8-76051508-generic'
make[1]: *** [Makefile:117: vmmon.ko] Error 2
make[1]: Leaving directory '/home/tom/Downloads/vmware-host-modules/vmmon-only'
make: *** [Makefile:21: vmmon-only] Error 2

tomarmistead
Contributor
Contributor

I commented out the ASSERT_ON_COMPILE macro in vm_asm_x86.h and was able to compile and install this.   It appears to be working fine now.

Not sure that it matters, but the CPU on my computer is AMD (instead of Intel); AMD Ryzen 7 5800X

Thanks for the fix!

vazquezm
Contributor
Contributor

Could you explain/show what did you exactly commnet out?

I have the same issue and I don't know what do I need to comment out.

Thank you

vazquezm
Contributor
Contributor

I figured out

I comment out the lines 67,68,69 in the file vmmon-only/include/vm_asm_x86.h

make

sudo make install

I this fixed the issue.

Thank you for the workaround

tomarmistead
Contributor
Contributor

Yes, exactly.  Sorry it took a bit for me to see this and respond. 

Here is how line 65-72 look after my edit to ./vmmon-only/include/vm_asm_x86.h

65 //#ifndef USE_UBSAN
66 //#define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr) \
67 // ASSERT_ON_COMPILE(sizeof(Selector) == 2 && \
68 // ((__builtin_constant_p(expr) && ((expr) >> 16) == 0) || \
69 // sizeof(expr) <= 2))
70 //#else
71 #define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr)
72 //#endif

 

After making this change;
make
sudo make install
sudo systemctl restart vmware

 

DigitalDude
Enthusiast
Enthusiast

@tomarmistead

BOOM!!!!  That did the trick.  And thank you for detailing the process!  Not everyone in the Land of Linux is a programmer.
 
 
 
Thanks, again.
sabennett
Contributor
Contributor

I just upgraded my Pop!_OS to 21.10 (regretting this now) and I'm receiving somewhat the same compile issue, instead of vmmon its vmnet

VMware compile fails with "Skipping BTF generation for /tmp/modconfig-9C2Ftg/vmnet-only/vmnet.ko due to unavailability of vmlinux"

Before finding this post I've uninstalled 16.1 and tried to install 16.2 (obviously that didn't work)

I've installed the headers with "apt install build-essential linux-headers-$(uname -r)"

root@lenovow540:/usr/lib# uname -r
5.15.8-76051508-generic

I tried the steps posted here but that didn't work for me. (likely because its vmmon and not vmnet)

Because I had been messing around trying other methods before I found this post, I decided to start over fresh again, so I reinstalled VMWare WS 16.2.

Now after the install there is only the 2 tar files for vmnet and vmmon, I extracted those tar files manually (tar xvf vmnet.tar then tar xvf vmmon.tar) and modified the vm_asm_x86.h in ./vmmon-only/ before doing the make and make install.

Now the 'Make' for vmmon went great no errors however the make install returned

"Using kernel build system.
make: *** No rule to make target 'install'. Stop."

I don't know if this is good or bad.  So I did what I do when my car makes a bad noise I turned up the radio so I couldn't hear it and moved on.

As far as the vmnet-only directory I'm not sure what file to modify.  The 'make' fails with:

Skipping BTF generation for /usr/lib/vmware/modules/source/vmnet-only/vmnet.ko due to unavailability of vmlinux
make[1]: Leaving directory '/usr/src/linux-headers-5.15.8-76051508-generic'

So sticking my head in the sand (again) I moved on with the following.

Before I found this thread I tried the method in this post here which didn't work because it doesn't have a step to comment out line 65-72 as you identified in file vm_asm_x86.h (located in ./vmmon-only).

But now I've had some success.

So I followed the last few step of the thread :

cp vmmon.o /lib/modules/`uname -r`/kernel/drivers/misc/vmmon.ko

cp vmnet.o /lib/modules/`uname -r`/kernel/drivers/misc/vmnet.ko

depmod -a

systemctl restart vmware.service

 

And well success.

I'll be honest, I have no idea what I've done or am doing but THANK-YOU!

Thanks @tomarmistead for the code edit and thanks @gdelcampo for a solution in another thread and thanks to @DigitalDude for having almost the exact same issue as me on the same platform and directing me to this great thread.

I don't know if this convoluted mess of a post will help anyone but there it is.

And thanks again.

🙂

Tags (3)
shibu_nyc
Enthusiast
Enthusiast

Can you or anyone post step by step instruction on how to build vmnet and vmmon yourself along with how to comment out the ASSERT_ON_COMPILE macro in vm_asm_x86.h?

Please assume I am an idiot former Windows user.

0 Kudos
sabennett
Contributor
Contributor

EDIT Update Dec30 2021:  Likely the methods shown in this post are better solutions than what I have posted here though this still works for 16.2.1 on Pop!_OS 21.10

---

I will try my best to clean up the steps and add more detail and linux commands

But first to make it clear I was not able to compile vmnet only vmmon after I modified the file vm_asm_x86.h found in the vmmon-only directory. I could not find the equivalent file in vmnet-only directory.

I'm not sure what the side effects may be by not having vmnet compiled correctly but for now everything seems ok on my Pop!_OS 21.10 install of VMWare WS 16.2

Also there are a couple differences in compile errors depending on where you are starting from "I think ...?"


I believe if you previously had a working copy of VMWare WS and then did the kernal upgrade only the vmmon failed to compile correctly.
However I uninstalled my VMware 16.1 and tried 16.2 hoping it would solve the compile issue and thus "I think" is the reason why I'm seeing the vmnet compile issue as well as the vmmon (I'm guessing totally guessing).
This procedure is written from the point where I've just done a fresh install of VMware WS 16.2

With that out of the way here are the detailed steps of updating vmmon

(vmnet would be almost the same with exception to whatever file would be modified and perhaps the directory structure)

Assumptions:
1. You have access to elevated priviledges meaning you can use the 'sudo' command
2. You have a perferred text editor like gedit, vim, leaf, etc installed
3. You have VMware Workstation installed.

4. You know how to open a terminal session

 

Steps to compile vmmon:
1. Navigate to the directory '/usr/lib/vmware/modules/source/'

I perfer the command line (CLI) to do this, open your terminal application and type cd /usr/lib/vmware/modules/source/

Like this: scott@lenovow540:~$ cd /usr/lib/vmware/modules/source/

 

2. Depending on your current VMware setup you may or may not see the directories vmnet-only and vmmon-only. If you're using the terminal use the ls command to see the directory contents. At the very least you should see two 'tar' files 'vmmon.tar' and 'vmnet.tar'

3. If the 'vmnet-only' and 'vmmon-only' directoy do NOT exist then use the following command with elevated priviledges in terminal 'sudo tar xvf vmmon.tar'
Your terminal screen should look something like this but the underline part will be your computer information and the bold is what you need to enter on the command line:

scott@lenovow540:/usr/lib/vmware/modules/source$ sudo tar xvf vmmon.tar

 

4. In the terminal type cd vmmon-only
You terminal prompt should look something like this: scott@lenovow540:/usr/lib/vmware/modules/source$ cd vmmon-only/include/

 

5. Use a text editor in elevated privilege mode (sudo). I believe gedit is preinstalled with ubuntu and pop!_os but any text editor will do such as vi or vim etc.
So using gedit command like this: scott@lenovow540:/usr/lib/vmware/modules/source/vmmon-only/include$ sudo gedit vm_asm_x86.h

 

6. Edit the vm_asm_x86.h file commenting out the compile issues from line 65 to 70 NOT 71 then 72. Just in case you're not aware, programmers place comments in their code, for example to explain what the following code function performs or to remove old code etc. Depending on the code there different methods to comment "out" a line of code. I think the code here is in "C" so any line with double "//" (not including the quotes) is ignored by the computer (compiler). Basically scroll down to line 65.

If you're using gedit the line numbers are on the left hand side. Make sure you add the double // on lines 65,66,67,68,69,70 and 72 as shown by @tomarmistead

Update EDIT: Originally I referenced earlier in my post commenting out line 65 to 72 (because lazy) tho I did identify the lines specifically to comment out plus referencing @tomarmistead original code change after but by then it's too late.

DO NOT COMMENT OUT LINE 71.  The line of code that says "#define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr)"

65 //#ifndef USE_UBSAN
66 //#define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr) \
67 // ASSERT_ON_COMPILE(sizeof(Selector) == 2 && \
68 // ((__builtin_constant_p(expr) && ((expr) >> 16) == 0) || \
69 // sizeof(expr) <= 2))
70 //#else
71 #define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr)     <<<---- DO NOT COMMENT OUT this line
72 //#endif

 

7. Save and close gedit.

 

8. Back in the terminal type: cd ..
Like this: scott@lenovow540:/usr/lib/vmware/modules/source/vmmon-only/include$ cd ..

 

9. In the terminal type: sudo make
Like this: scott@lenovow540:/usr/lib/vmware/modules/source/vmmon-only$ sudo make

 

10. In the terminal type: cd ..

 

11. You should be back in the sources directory
Like this: scott@lenovow540:/usr/lib/vmware/modules/source$

 

12. NOTE: Now its rinse and repeat minus the file edit because I'm not a coder and don't know what file or what to change for the vmnet. That said I believe you still need to do the 'make' even though it doesn't complete successfully. The next steps look like this from the terminal (CLI)...

 

13. sudo tar xvf vmnet.tar


14. cd vmnet-only


15. sudo make


16. cd ..


17. sudo cp vmmon.o /lib/modules/`uname -r`/kernel/drivers/misc/vmmon.ko


18. sudo cp vmnet.o /lib/modules/`uname -r`/kernel/drivers/misc/vmnet.ko


19. sudo depmod -a


20. sudo systemctl restart vmware.service

 

21. Start VMware WS from the graphical interface. Clicky the icon 🙂

 

Sorry I don't know a way to do all this from a graphical interface point of view (aka windows) specifically the "make" function.

Good luck I hope this works for you.  I have a second machine running Pop!_OS and repeated these steps and it worked for me the second time.

 

mkubecek
Hot Shot
Hot Shot

For the record, this looks rather like a distribution compiler bug than a problem with vmmon driver code.

ehfl
Contributor
Contributor

Are you on a System76 machine? I am and having the same issue. I'm running Ubuntu, not Pop!_OS but I'm wondering if it has to do with the System76 driver/System76 preferred kernel?

0 Kudos
robertmglynn
Contributor
Contributor

Thanks for the effort in this. However, I'm unable to get past an error preventing make from succeeding:


/usr/lib/vmware/modules/source/vmmon-only/./include/vm_asm_x86.h:263:7: error: implicit declaration of function ‘ASSERT_ON_COMPILE_SELECTOR_SIZE’ [-Werror=implicit-function-declaration]
263 | ASSERT_ON_COMPILE_SELECTOR_SIZE(expr); \

which makes sense, since the "fix" comments out the definition of the ASSERT_ON_COMPILE_SELECTOR_SIZE function.

I'm on PopOS 21.10, which is basically Ubuntu 21.10

Can't find any help from VMWare directly; still down...

0 Kudos
tomarmistead
Contributor
Contributor

@ehfl  I'm on a System76 system, but using Pop OS!_21.10

0 Kudos
tomarmistead
Contributor
Contributor

@robertmglynn you don't want to comment out the macro definition, you want to comment out all except for the empty definition of it.

If you comment out the full definition, make sure this remains;
#define ASSERT_ON_COMPILE_SELECTOR_SIZE(expr)

robertmglynn
Contributor
Contributor

@tomarmistead @ehfl I forgot it in my comment; I'm also on a System76 Darp6, VMWare Workstation Pro stopped working immediately after upgrading to PopOS 21.10.

0 Kudos
tomarmistead
Contributor
Contributor

@robertmglynn VMware  worked after my initial upgrade to 21.10,  but broke after the first update was applied to it...
With the above changes, it's working now.   Unless VMware comes up with a new update, this "fix" will need to be applied after any update that contains a new kernel.