topic Re: unable to run openshift on a RHEL 9.1 virtual machine in VMware Fusion Discussions

unable to run openshift on a RHEL 9.1 virtual machine

mariocloud — Wed, 12 Jul 2023 16:25:17 GMT

Hi There,

I am running Red Hat Enterprise Linux 9.1 in vmware Fusion 13.01 on Mac OS Ventura 13.3 x64 on a Mac pro 2019

Processor Name: 8-Core Intel Core i9
L2 Cache (per Core): 256 KB
L3 Cache: 12 MB
Hyper-Threading Technology: Enabled
Memory: 16 GB

The kernel version of the RHEL9.1 vm is:
5.14.0-162.22.2.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Mar 15 14:44:24 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux

When I am trying to run openshift on the Red Hat virtual machine I am getting:

crc start
Error starting machine: Error in driver during machine start: virError(Code=1, Domain=10, Message='internal error: qemu unexpectedly closed the monitor: 2023-07-12T17:28:47.805718Z qemu-kvm: error: failed to set MSR 0x48d to 0x5600000016
qemu-kvm: ../target/i386/kvm/kvm.c:3013: int kvm_buf_set_msrs(X86CPU *): Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.')

Did anyone manage to address this already?

Thanks a lot in advance..
Ciao,,Mario.

Re: unable to run openshift on a RHEL 9.1 virtual machine

Mikero — Wed, 12 Jul 2023 16:53:27 GMT

Sounds like the guest is trying to do something with QEMU. You'll need nested virtualization enabled for that. (VM Settings > Advanced > Enable Hypervisor applications...)

Re: unable to run openshift on a RHEL 9.1 virtual machine

mariocloud — Wed, 12 Jul 2023 18:57:08 GMT

Hi Mikero,

sorry, I missed to specify that I already enabled "Hypervisor applications in this virtual machine"... and unfortunately that

issue is there..

Thx a lot for the reply..
Ciao..Mario.

Re: unable to run openshift on a RHEL 9.1 virtual machine

Mikero — Wed, 12 Jul 2023 19:22:47 GMT

Ah, got it.

So I looked a bit and I'm more confused now lol

The error is similar to ones we saw when Apple updated macOS to require Hypervisor.framework (which we rely on) to not work with Macbook Pros without VMCS shadowing / VPRO when trying to run something nested (i.e. ESXi, QEMU, KVM, Hyper-V, VMware Workstation...)... However I'd expect the Xeon in the 2019 Mac Pro to have that.

Are there any errors in the vmware.log of the VM?

Re: unable to run openshift on a RHEL 9.1 virtual machine

mariocloud — Wed, 12 Jul 2023 21:28:20 GMT

Hi Mikero,

I'm sorry but I cannot get the vmware.log file.. Could you please share any info about where it could be located..?

Thx a lot again..
Ciao...Mario.

Re: unable to run openshift on a RHEL 9.1 virtual machine

Mikero — Wed, 12 Jul 2023 21:35:51 GMT

log files for the vm process are located within the VM bundle.

Go to the VM in Finder, right-click it, click "show package contents.", vmware.log should be in there

Re: unable to run openshift on a RHEL 9.1 virtual machine

dempson — Wed, 12 Jul 2023 23:05:28 GMT

Given the symptoms and one other key detail mentioned, I'm pretty sure the original post has the name of the computer wrong: a 2019 Mac Pro would have a Xeon processor, not an "8-core Intel Core i9". A 2019 MacBook Pro would.have the latter processor.

There are two variants of the Core i9 used in 2019 MacBook Pros (mid 2019 15-inch and late 2019 16-inch).

2.3 GHz 8-core Core i9 (9880H)
2.4 GHz 8-core Core i9 (9980HK)

The 2.3 GHz was the standard processor in the more expensive base model. The 2.4 GHz was a custom upgrade for any base model.

Of the two, the cheaper 2.3 GHz Core i9 (9880H) supports vPro and VMCS shadowing, but the more expensive 2.4 GHz Core i9 (9980HK) does not.

One other detail in the original poster's description doesn't match: the two above both have 16 MB of L3 cache, but the original post said 12 MB.

The previous 2018 15-inch MacBook Pro (sold new until May 2019) had a custom option for a 2.9 GHz 8-core Core i9 (8950HK) and 12 MB of L3 cache. That one doesn't do vPro or VMCS shadowing either.

Of the three possible Core i9 processors, only the 2.3 GHz one in the 2019 15-inch and 16-inch MacBook Pro can do nested virtualisation with Apple's hypervisor in VMware Fusion.

(The Xeons in the 2019 Mac Pro start at 8-core with 24.5 MB of L3 cache and go up from there. All of them support vPro and VMCS shadowing.)

Re: unable to run openshift on a RHEL 9.1 virtual machine

Mikero — Wed, 12 Jul 2023 23:09:00 GMT

Ah, bingo... nice catch!

Re: unable to run openshift on a RHEL 9.1 virtual machine

mariocloud — Thu, 13 Jul 2023 08:22:41 GMT

Hi Mikero,

Thx a lot for your deep analysis..
As per attached screen-shot, my Mac has on board the 2.3 GHz and so (based on your findings) can do nested virtualisation with Apple's hypervisor in VMware Fusion.

As a reference I attached also the vmware.log file.

Thx a lot again.. I really appreciate.
Ciao..Mario.