rgadsdon
Enthusiast
Enthusiast

Linux Host : Kernel 5.8-rc1/rc2. vmware-modconfig segfaults. vmware runtime causes system reboot when Guest VM start is attempted

On x86_64 systems, Fedora 32, VMware Workstation 15.5.6 with latest vmmon/vmnet patches.  

Host system Kernels tested are 5.8-rc1 and 5.8-rc2 (kernel.org), and Fedora 5.8.0-0.rc1.20200617git69119673bd50.1.fc33.x86_64

vmmon/vmnet with patches compile and load OK.

vmware-modconfig segfaults:

# vmware-modconfig –console –install-all
[AppLoader] GLib does not have GSettings support.
vmware-modconfi[14782]: segfault at 0 ip 00007f786b9d47f7 sp 00007ffc52a36cf8 error 4 in libc-2.31.so[7f786b95a000+150000]
Code: Bad RIP value.
Segmentation fault (core dumped)

vmware runtime segfaults:

$ vmware
/usr/bin/vmware: line 105: 13529 Segmentation fault (core dumped) “$BINDIR”/vmware-modconfig –appname=”VMware Workstation” –icon=”vmware-workstation”

Run vmware binary directly, and VMware Workstation window/menu displays correctly

# /usr/lib/vmware/bin/vmware

Start up guest operating system, then after ‘waiting for connection‘ the host system immediately reboots, with no prior errors or warnings shown. 

This has been reproduced on three different systems (HP Z220, HP 800G1, and Dual Xeon E5-2687W)

Robert Gadsdon.  June 23rd 2020.

13 Replies
Mits2020
Hot Shot
Hot Shot

As per the latest VMware Knowledge Base, Workstation 15.5 supports hosts up to Fedora 30. This document is dated 10/12/2019 and perhaps a newer version exists.

rgadsdon
Enthusiast
Enthusiast

From further investigation, including info from certain other VM technologies, it would appear that changes in the Linux 5.8 kernel will prevent VMware hosts from operating, in their current form, and there is no practical solution available at present..?

If anyone has more useful information on this, bearing in mind that Kernel 5.8 will be released in a couple of weeks time, it would be helpful...

Robert Gadsdon.    July 21st 2020

0 Kudos
mkubecek
Enthusiast
Enthusiast

Cannot do much about the segfault, obviously. Tried to look at the crash but kdump is not triggered and there is nothing helpful on serial console either which indicates either a triple fault or something messing with the hardware in a bad way. Bisect might give us some clue but that would be quite time consuming and I cannot afford to spend that much time on this issue at the moment (and I'm not even sure this is something that can be resolved by only patching vmmon/vmnet source).

0 Kudos
jkwhite
VMware Employee
VMware Employee

The vmware-modconfig segfault is a result of a format change in /proc/version.

If it helps for local patch creation, it came in from the change at kernel/git/next/linux-next.git - The linux-next integration testing tree

The system reboots are somewhat more trouble to deal with - a result of

kernel/git/next/linux-next.git - The linux-next integration testing tree

The fix for the second item is fairly extensive and not just to the vmmon module so there may not be a clean workaround short of reverting that patch from the kernel (until a fixed WS build is available).

0 Kudos
mkubecek
Enthusiast
Enthusiast

Thank you for the info. I was afraid it would be the W^X stuff as (1) virtualbox also ran into issues with this and (2) a triple fault usually means messed up page tables. I guess this also means game over for people still using 14.1.7 or 12.5.9 but that was bound to happen one day.

0 Kudos
lipnitsk
Contributor
Contributor

Do you have an ETA on the fixed WS build?

0 Kudos
jkwhite
VMware Employee
VMware Employee

Other than "in WS 16.0" I don't have an any more specific date handy.

The final round of fixes just missed the 20H2 tech preview that went out in July.

0 Kudos
Rakasa
Contributor
Contributor

Very disappointed VMware seems to be so far behind... the most recent Fedora listed as supported is 30, and that's already EOL!

zephhull
Contributor
Contributor

Has the issue with the change to /proc/version been reported upstream?

This feels like the kind of breaking change that usually gets reverted pretty quickly.

The memory mapping permissions change is probably going to be somewhat more difficult though.

0 Kudos
mkubecek
Enthusiast
Enthusiast

You can try but I don't think it's really appropriate. Unlike the missing trailing null character in /proc/self/cmdline in 4.18-rc1, this is a minor format change in a text file intended rather for user's information than for parsing. Also, there is quite a difference between parser failing to find the information (gcc version?) and parser segfaulting on unexpected input.

0 Kudos
mukmuk302
Contributor
Contributor

I know this is old, but we need a fix for this.

I use(d) VMware Workstation daily for work (currently on 15.5.7). No budget to upgrade to 16. I'm dead in the water.

I don't have any previous kernels to boot too. The only option left is to wipe my host and install an old release. That's a dumb workaround for a paid product.

Please fix!

Thanks.

 

bestfind
Contributor
Contributor

0 Kudos
jimblue1
Contributor
Contributor

I have to use VMware 15.5.6 because a patch is available that works with older kernel 5.4 

VMware patch: https://github.com/mkubecek/vmware-host-modules/tree/workstation-15.5.6

My setup:
Ubuntu 20.04.2 LTS x86_64
Kernel: 5.4.117-0504117-generic
VMware® Workstation 15 Pro 15.5.6 build-16341506

If you really need kernel 5.8 then just upgrade to VMware 16.

0 Kudos