topic Key was rejected by service in VMware Workstation Pro Discussions

Key was rejected by service

pgaltieri — Wed, 03 Feb 2021 18:31:35 GMT

I have installed Workstation Pro 16.1 on Fedora 33. It fails to start. After some investigating it comes down to this

modprobe vmmon
modprobe: ERROR: could not insert 'vmmon': Key was rejected by service

The same happens for the vmnet module.

running dmesg shows

vmmon: Loading of unsigned module is rejected

I have Workstation Pro 16.1 running on Fedora 31 and Fedora 32 and it works fine there. So obviously something has changed in Fedora 33 to cause this problem. So what do I need to do to fix the problem?

Any help is appreciated

Paolo

Re: Key was rejected by service

wila — Thu, 04 Feb 2021 13:33:31 GMT

Hi,

The kernel module is not signed with a certificate and if you want to use it on fedora 33 then you -apparently- have to sign it.

More info on how-to do that is here:
https://docs.fedoraproject.org/en-US/Fedora/23/html/System_Administrators_Guide/sect-signing-kernel-modules-for-secure-boot.html

Good luck!
--
Wil

Re: Key was rejected by service

pgaltieri — Thu, 04 Feb 2021 17:07:42 GMT

Wila,

thanks for the information. I was wondering though how does it work on Windows? This laptop boots to both Windows and Fedora and Workstation 16.1 runs fine on Windows?

Paolo

Re: Key was rejected by service

wila — Thu, 04 Feb 2021 17:53:46 GMT

Paolo,

On Windows signing kernel extensions is a thing for many years. It has been required to do since at least Windows Vista. On windows the kernel extensions come pre-built by VMware and are also signed by VMware.

On Linux however, the kernel is not exactly stable in its programming interface and as a way to help you, VMware provides the source of the kernel modules. So you compile these yourself. As such they cannot provide the certificates from VMware to sign.
Over time I expect VMware to handle this as well, by generating the internal/local certificates on your host and use those to sign the kernel extensions after you built the modules.

Long story short, this requirement for signing Linux kernel modules is fairly new and as such VMware does not have a "just click to upgrade" solution for this.

The admin guide I pointed to basically holds you by the hand on how-to do this yourself over about 5 pages (click Next at the bottom to continue)

I'm sure that there's some blog posts out there that explain this in more details as well.

--
Wil

Re: Key was rejected by service

Learner4 — Fri, 05 Nov 2021 04:58:57 GMT

Thanks for the info.
Any links on how to get VMWARE player to work on OpenSUSE 15.3. We followed the instructions for making MOK keys but get same error as the pgaltieri.

When we check the modprobe for vmmon, it shows
modprobe: ERROR: could not insert 'vmmon': Key was rejected by service

We followed these steps -> vmware kb2146460

Any suggestions would be helpful....