VMware Communities
WTell
Contributor
Contributor

Abysmal Performance in Fusion 12 after upgrade to macOS Big Sur

After upgrading to macOS Big Sur, performance on my VMs went from great to utterly abysmal. Remedial tasks like opening the Start menu, opening Notepad etc. can now take up to half a minute. Windows are drawn on the screen line-by-line with half second intervals.

The host computer is a 2018 Mac mini with ample memory, disk and CPU. The issue was not present under Fusion 12.0.0 before upgrading to macOS Big Sur. Toggling "Disable Side Channel Mitigations" has no effect.

The attached screenshot shows the vmx processes of three VMs after the VMs have been idling overnight, with no applications open. The VMs (running Windows 10) are pegging the CPU while doing nothing.

Screenshot 2020-11-17 at 08.07.53.png

 

Tags (1)
78 Replies
mlippold1
Enthusiast
Enthusiast

I had the same exact thing happening before I gave up and rolled back to Catalina (which has always run perfectly).

I have a 2019 27" iMac with the highest CPU/Memory and Video card I could choose when purchasing it.   The CPU has VMCS shadowing support so that isn't the cause.  I also turned off the side channel mitigations and it did nothing.   I use Virtualization Based Security on my VMs which I require for work so I can't uncheck the hypervisor or anything like people have been suggesting.

There is something seriously wrong with VMWare's implementation with Big Sur.

ColoradoMarmot
Champion
Champion

What's your host and guest config?  Very important that the guest has no more than n-1 cores assigned, where n is the physical cores (not virtual or hyper threaded) on the host.  Also, Big Sur seems to need more ram to function well...still experimenting, but it looks like 4gb for host is the optimal minimum.

Reply
0 Kudos
nobody456
Enthusiast
Enthusiast

Similar story for me. Fusion 12 on Catalina was fine. After upgrading to Big Sur, VMs ran at something like 1/50th speed, i.e super slow startup, login, anything.

 

Disabling the side channel mitigations seemed to help -- now it feels like maybe 95% of prior performance, i.e. I can still tell a slowdown (mostly via muscle memory).

 

Big Sur has been a Big Letdown so far.

 

So. Much. Whitespace. (And yes, the vertical pixel height of my VMs required adjustments.)

So. Many. Round Corners. (And yes, the lower left and right corners of my VMs are clipped.)

So. Much. Change. For. The. Sake. Of. Someone. At. Apple. Getting. Promoted. (Random example: getting used to totally different sound effects... may seem cute to the random @#$%^&* at Apple who insisted on new sounds, but is a major disruption to my mental workflow.)

 

Overall... maybe the transition to M1 is a good thing after all... because it forces me to migrate off Apple, once and for all -- staying with the VMs is, at least for me, more important than staying with MacOS.

 

So yeah... thanks VMware... you've been good to me.  🙂

Reply
0 Kudos
WTell
Contributor
Contributor

I have two identical Mac minis (2018) and the issue is only present on one of them. Very strange indeed.

@mlippold1 I'm exactly where you are. I can't turn off VBS because I need it for my work.

@ColoradoMarmot Thanks for your input. My host has 6 physical cores, while my VMs have between 2 and 4 processor cores assigned to them. My host has 32 GB RAM, and I've experimented with assigning low or high amounts with no success. On the one Mac mini where performance is excellent, performance is still great when I assign all 6 cores.

@nobody456 What did you get most out of – enabling "Disable Side Channel Mitigations" or disabling "Enable hypervisor applications in this virtual machine"? For what it's worth, I cannot agree more with your other sentiments about Big Sur. I cannot fathom why I now have to live in perpetual loss of the lower left and lower right corners of my VM screens. I also have a lot of trouble identifying which window is on top (focus) or not – the difference in brightness/contrast is so negligible that I feel like I'm suddenly visually impaired.

nobody456
Enthusiast
Enthusiast

What did you get most out of – enabling "Disable Side Channel Mitigations" or

> disabling "Enable hypervisor applications in this virtual machine"?

I had the latter disabled.

The former is what I was offered for the first time tonight (i.e. Fusion didn't give me a pop-up window for it under Catalina). And selected.

I haven't tried disabling it again -- maybe when I have more spare time.

> [Big Letdown] I feel like I'm suddenly visually impaired.

This. Exactly this.

My computers are supposed to make my life easier, not harder.

This MacOS version utterly fails to deliver on that -- it's a big step back.

Apple... eh, whatever... my personal long term trajectory is clear to me.

 

WTell
Contributor
Contributor

Update: I think I have found what makes this bug materialise. At least, I can now consistently reproduce my issue.

See the attached screenshot. When the VM setting "Enable hypervisor applications in this virtual machine" is enabled, the issue is only present when "Memory integrity" is turned on inside the VM. Turning "Memory integrity" off vastly improves responsiveness, performance and brings the host CPU load under idle to reasonable levels.

With "Memory integrity" turned on, the VM was idling at ≈390% CPU use on the host. Opening "My computer" inside the VM from a blank desktop took 9 seconds. This bug was not present before upgrading to macOS Big Sur.

With "Memory integrity" turned off, the VM is idling at ≈6% CPU on the host. Opening "My computer" inside the VM is practically instantaneous.

Screenshot 2020-11-18 at 08.09.44.png

 

nobody456
Enthusiast
Enthusiast

My guest Windows predates "Memory Integrity".

By more than a decade.

So that alone isn't gonna be the culprit here.

Reply
0 Kudos
wila
Immortal
Immortal

Hi,

nobody456 said:
"My guest Windows predates "Memory Integrity".
By more than a decade.
So that alone isn't gonna be the culprit here."

Something comparable might still be causing the same thing.
Are you running an antivirus in the guest? If so check this old thread:
https://communities.vmware.com/t5/VMware-Workstation-Player/XP-VM-suddenly-slow-Win-7-fine/m-p/22516...

hope this helps,
--
Wil

| Author of Vimalin. The virtual machine Backup app for VMware Fusion, VMware Workstation and Player |
| More info at vimalin.com | Twitter @wilva
Reply
0 Kudos
mlippold1
Enthusiast
Enthusiast

My machines run perfectly in Catalina - I've had 5+ different machines running beautifully at a time but I usually have only three up.

The three I have up all require that I use Virtualization Based Security which seems to be the problem in Big Sur (any sort of nested virtualization).

The three machines are:  4 cores + 8GB, 4 cores + 8GB and 6 cores + 10GB.

I have an 8 core i9 (16 logical cores total due to hyperthreading being enabled - In vmware it also reflects that my processor pool is 16 cores). 

My machine:

iMac (Retina 5K, 27-inch, 2019)

Processor: 3.6 GHz 8-Core Intel Core i9

Memory: 64 GB 2667 MHz DDR4

Graphics: Radeon Pro Vega 48 8 GB

As others had stated as well - Even if I leave a few VMs up idling for a day it was just using like 80% of my Mac's processor.   Something is wrong with Apple's hypervisor or VMware implementation in using it.   Apple has probably just created a terrible hypervisor "middle-man" that is extremely bad at its job.  

 

Reply
0 Kudos
mlippold1
Enthusiast
Enthusiast

Thanks for this idea at least - With the pain it took going back to Catalina from Big Sur (My Time Machine restore wasn't working right, I had to create a bootable USB key and obtain the Catalina install app, which of course wasn't working per Apple's instructions and then install Catalina again and then do my Time Machine restore again.).  One day, when I'm not annoyed with Apple, I may go back and try this.

 

I feel like maybe this is just doing the same thing that VMWare is doing with having you turn off the side channel stuff.  Perhaps it isn't allowing the VMWare change from working though.  Worth a shot at some point.

 

* Actually, I just looked and it isn't even something I can shut off due to the group policies that are required for my machines by my employer.

Reply
0 Kudos
ColoradoMarmot
Champion
Champion

Ignore logical cores.  You have an 8 core machine, so if you're running all three at once, you're overloading the host for sure.

 

But separately, do you have a fusion drive?

Reply
0 Kudos
WTell
Contributor
Contributor

@ColoradoMarmot Thanks a lot for your input. This is clearly not a resource congestion/exhaustion issue, as the performance hit is almost hundredfold when "Memory integrity" is turned on and resources are plentiful. Since the issue arose with the move to macOS Big Sur, I'm forced to conclude that this somehow is related to the changes to the Hypervisor Framework introduced with the macOS 11 SDK. Either Apple or VMware must address this.

Reply
0 Kudos
ColoradoMarmot
Champion
Champion

Yeah, I responded before catching that piece of the thread.  The new framework is both a blessing and curse for sure.  At least you have a workaround temporarily.

Reply
0 Kudos
grrowl
Contributor
Contributor

Not sure if this will help others, however with a WinXP VM I traced it down to the VMWare tools video driver. When I removed the VMWare SVGA driver and used the default driver performance was much improved.

Reply
0 Kudos
vandaveer
Contributor
Contributor

Hello,

I found this thread because one, and only one, of my VMs was performing terribly since upgrading first to Big Sur and Fusion 12. Actually, that's an understatement. It was completely unusable. Like some other respondents, I too am running on a Mac Mini and in my case with the best CPU, 2 TB SSD, and 64 GB of RAM. No possibility for resource contention. What resolved the issue for me was turning off the hypervisor, Virtual Machine -> Settings -> Processors and Memory -> Enable hypervisor applications in this virtual machine. A few items to note: I did not have core isolation turned on in my VM so that was not the issue for me, I have not changed the compatibility up from 16 yet in that VM. I have another VM that is just as fast as ever that is running on 18 and has the hypervisor settings on. I know that none of this is conclusive and unfortunately I don't have time to do some A/B testing but hopefully that info helps someone.

nobody456
Enthusiast
Enthusiast

with a WinXP VM I traced it down to the VMWare tools video driver. When I removed the VMWare SVGA driver and used the default driver performance was much improved.

keeping the tools up-to-date tends to be a good idea in general

however, the tools are not the culprit for the Catalina -> Big Sur slowdown: same Fusion and tools version before and after; only MacOS changed

Reply
0 Kudos
nobody456
Enthusiast
Enthusiast

What resolved the issue for me was turning off the hypervisor, Virtual Machine -> Settings -> Processors and Memory -> Enable hypervisor applications in this virtual machine.

I don't think that's the true culprit either though

in my case, that setting has been disabled forever, because the guest OS (W2K) predates the arrival of x86 virtualization extensions/support

to re-cap: Fusion 12, W2K guest, latest tools, "enable HV apps in this VM" never enabled – fine under Catalina, then slow after update to Big Sur, then fine again after "side channel mitigations" disabled (as prompted by Fusion with a msg box, upon first VM startup after the MacOS update – again, no Fusion, tools, VM etc. update; only MacOS update)

and as I mentioned, the straightforward Fusion logs for the VM show no notable difference between "side channel mitigations" enabled (slow) and disabled (fast) – so everything hints at the ball being in Apple's court, not VMware's court... most likely with Apple's user-mode virtualization framework that hypervisors like Fusion were forced to switch to

if I have more spare cycles, then I might debug it further; no promises though

running with "side channel mitigations" disabled... well... you do need to make your own decision on that, of course, based on your particular risk profile...

Reply
0 Kudos
nobody456
Enthusiast
Enthusiast

one more tidbit:

 

for me, it's a 2-core VM, which I typically boot with just 1 core (via the W2K boot command line option)

 

with "side channel mitigations" enabled (aka slow), it hovers at about 400% load – so yeah, core isolation at its worst: allocate 2 physical cores of the host Mac, and peg their hyper threads too... leading to 2*2 = 4 --> 400%

 

with "side channel mitigations" disabled (aka fast), it hovers at about 40% load when the VM is sitting there idle, plus another 20% or so for the MacOS WindowServer (doing its normal thing i.e. drawing stuff onto the screen)

Reply
0 Kudos
mlippold1
Enthusiast
Enthusiast

@ColoradoMarmot 

If you are talking to me you are just plain wrong.  With my three VMs up and running my CPU just sits at 10 percent most of the time in Catalina.    I have an external Thunderbolt enclosure with 4 512GB NVME M.2 SSDs that run at the max speed of Thunderbolt 3 so there is no I/O bottleneck.  In Big Sur I can start one VM with VBS enabled and it will peg my CPU at 80 percent and never go down.   Big Sur's Hypervisor (or VMWares implementation of it) is unusable if you have nested VM type operations right now.