Fusion 12 Pro No Nested Virtualization

[johnmaenpaa]

We were assured the GA version of Fusion 12 would support nested virtualization in Big Sur. It does not appear to.

[RDPetruska]

...And Big Sur is still in beta form at the moment!  So I'd wait until it is GA.

[Mikero]

It's more complicated unfortunately.

It turns out the virtualization APIs that were provided to us to support this use case require a specific Intel CPU hardware feature to function. (VMCS Shadowing).

The unfortunate part is that this hardware feature is not commonly found on consumer oriented CPUs. (None of my MacBook Pros support this anymore... I'm staying on Catalina on most of my machines as a result).

We're working with Apple to address this regression, as it would need enhancements to macOS itself.

[hkang]

VMware Knowledge Base has more details about nested limitations on Big Sur.

[gringley]

My Googling basically brought up stuff from 2013 and 2014 for VMCS Shadowing.  How does one confirm if their Mac's CPUs might have it?

[Mikero]

The easiest way I've found thus far is by looking at the vmware.log of a VM that has VT-x enabled. (this can be done when the VM is running)

From the VM Library, select the VM > right click > hold 'Command' > Click 'Open Latest Log File' (neat little trick...)

From Console.app, search 'vmcs' and you should see an entry like the following:

<timestamp> vmx| I005:   Use VMCS shadowing                       { 0 }

This is my 2019 MBP, and it does NOT have VMCS shadowing.

This is from my 2017 iMac Pro, which does have VMCS shadowing:

<timestamp> vmx| I005:   Use VMCS shadowing                       {0,1}

[SvenGus]

I tried this on a late 2013 MBP (Core i7 Haswell) with Catalina 10.15.6 and Fusion 12 Pro, and the output was this (with double entries: {0,1} and { 0 }😞

vmx| I005:   VMCS revision ID                  18

vmx| I005:   VMCS region length              1024

vmx| I005:   VMCS memory type                  WB

vmx| I005:   Use VMCS shadowing                       {0,1}

vmx| I005: VMCS Enumeration (0x000000000000002a)

vmx| I005:   VMCS revision ID                   1

vmx| I005:   VMCS region length              4096

vmx| I005:   VMCS memory type                  WB

vmx| I005:   Use VMCS shadowing                       { 0 }

vmx| I005: VMCS Enumeration (0x000000000000005a)

vmx| I005:    VMCS shadow

vmx| I005: OvhdMem OvhdUser_vhvCachedVMCS              :       2      2      - |      2      2      -

vmx| I005: OvhdMem OvhdUser_vhvCachedVMCS              :       2      2      - |      2      2      -

... So, does it support VMCS shadowing, or not...?

[bluefirestorm]

If the VM has "Enable hypervisor applications in this virtual machine" checked, there should be two sections.

vmx| I005: Host VT-x Capabilities:

vmx| I005: Guest VT-x Capabilities:

So the one to look at is the "Use VMCS shadowing" under the Host VT-x Capabilities which is the one that comes first.

From the Intel paper referenced in the KB, it looks like vPro Eligibility "Yes" in the Intel ARK website would indicate whether VMCS Shadowing is present or not for Haswell and newer CPUs.

This link is a comparison of all 2013 MBP with i7 CPUs.

https://ark.intel.com/content/www/us/en/ark/compare.html?productIds=75992,76086,76087,76088

[gringley]

...I knew I got that iMac Pro for something...and now I know!  Thanks!  I was able to confirm the log entries.

[SvenGus]

In nested virtualization, is Apple’s hypervisor perhaps still much slower than VMware’s...? That would explain some things in Big Sur; or maybe there are other reasons (side channel

mitigations? a Big Sur-only problem, thus?)...

[SvenGus]

After some more experiments, at least on my machine (old, but apparently capable of doing nested virtualisation on Big Sur), nested virtualisation is essentially unusably slow on Big Sur, even with side channel mitigations disabled. Let's hope this situation will evolve...

[Mikero]

What hardware do you have?

[SvenGus]

MacBook Pro 15-Inch "Core i7" 2.3 Late 2013 (DG) Specs (Retina Late 2013 15", ME294LL/A, MacBookPro1...

[SvenGus]

... So, it looks like these older Macs indeed support VMCS shadowing, as everything works without error messages; but while, for example, in a Catalina host booting a Windows 7 installer takes only a couple of minutes inside a nested VMware Player 16 in a Windows 10 Fusion 12 VM, in a Big Sur host it still doesn’t reach the “install” stage after an hour: so something is evidently wrong somewhere, even if formally it “works”; and turning off side channel mitigations only speeds things up a little at the beginning: afterwards, after the “Starting Windows” phase and reboot, it seems to hang indefinitely, as described before. A rather curious behaviour, anyway...

[s_a_]

Hello,

I know your post is almost 6 month old but I have upgraded my Mac to Big Sur recently, and my VMWare Fusion Pro 12.1.0 Nested Virtualization stopped working, I actually had to disable the hypervisor setting to be able to run the VM. It looks like the Nested Virtualization problem on Big Sur is still happening.

My brand-new MacBook CPU doesn't support for VMCS Shadowing.

Well, actually I can start my VM with nested virtualization turned on, however it's so slow that I can't work on it, it's unusable slow.

Is there an ongoing activity to make VMWare Fusion Professional Version 12.1.0 (17195230) version to work on Big Sur with nested virtualization?

Do you guys have any update about it?

thanks in advance

[mlippold1]

I wish we'd get an answer for this as well but I've heard exactly zero from anyone from VMWare on this.