VM became extremely slow after upgraded to macOS 10.14.6 [Officially Solved]

[Luminatio]

------------20190827--------------------------

This issue is officially solved with Apple’s macOS 10.14.6 supplemental update.

Thanks to all of you.

A thumb down to Apple’s software team...

————————

After I upgraded my 2 MBP to 10.14.6, all of the Windows VMs became extremely slow and stuck at the login screen...

The VMs are set to use TPM, I don't know if it's relevant.

Anyone came across the same situation?

Now I cannot continue my work...really frustrating...

—————20190727————-

I’m updating this for convenient reference.

Status: This issue has not been solved yet.

Many users encountered this issue which leads to completely unusable of VMs.

And we got some replies from Fusion developers, you could help developers identify issues by replying your hardware models/is disk encrypted or have the VM encrypted etc.

We have some reports that some users found out that if set the memory of VM to 2G could help.

Re: VM became extremely slow after upgraded to macOS 10.14.6

If you don’t need to use much memory and can work with it, you might try set the memory to 2G, or maybe less than 4G is also okay?

Still we are waiting for an official solution.

------------20190809--------------------------

Thanks to ksc

Re: VM became extremely slow after upgraded to macOS 10.14.6

I updated to Fusion 11.1.1, confirmed that I can login to the system, and desktop shows normally.

As ksc says, it is a workaround which I'm not quite sure if I can trust it to do my work in it....

At least, I'm able to get my works out and change to other options now.

I'll mark this issue as resolved for now, but keep the [Workaround Released] tag in the subject.

Looks like it would take quite some time for a real fix, I'll keep an eye on this.

消息编辑者为：Liam

[bcjacob]

I have the same issue.

MacBook Pro (Retina, 15-inch, Mid 2015)

encrypted disk

vm gues is Windows 10

encrypted disk with TPM.

System interrupts 100% cpu (took 15 minutes or more just to get task manager to actually open)

Accessibility suggestion is not working for me.  Windows is completely locked and unusable.

[bolaz]

It doesn't work for me (windows 10 64-bit encrypted)

[mcapon]

Can we get a response/update from VMware?  This is very frustrating, and silence from VMware is not helping.

[pauljacobevans]

Got one better...

My VM barfed post-update to 10.14.6 with something like "can't find open peer?" or something?  Never was able to even get the VM started... It would routinely attempt to restore the state and after an hour (3 attempts) progress seems staginated.

So f-it... Let's blow it away and reinstall Windows!

Oh...

Can't even install Windows 10!  VM Seems fine, slugs down... and now my final attempt it is completely ham-fried at "Getting files ready for installation (52%)". Several attempts, nothing seems any better.

Encrypted HD with Vmware TPM.  Migrated over the VMWare specifically for that setup.

Seriously?

MBP 2017

macOS: 10.14.6
Fusion 11.1.0

[ksc]

Official reply... I don't have answers, but can at least say what's being done.

First - I'm the developer looking directly at this. And if you look at my forum profile - I've been around a long time, as a core developer. Fusion isn't a forgotten product.

The bad news is, it's very puzzling. What we know:

• Encrypted VMs are hit worse than unencrypted VMs (which are also impacted).
• 10.14 and 10.15 betas are impacted. 10.13 is not impacted.
• There is a consistent pattern of I/O that looks good for a while, then gets really bad. Where "bad" means "a spinning disk would be faster".

If you want to confirm this slowdown after some amount of I/O, enable performance monitoring (Settings -> Advanced -> Troubleshooting, select "Performance"). In addition to profiling and statistics collection, this mode enables some log messages... some of which look like this:

2019-07-25T19:50:38.877-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=446us pavg=528us n=16384 d=0 i=16384 vmx=99.5% 16300

2019-07-25T19:50:50.741-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=442us pavg=408us n=18432 d=0 i=18432 vmx=99.5% 18348

2019-07-25T19:51:01.047-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=423us pavg=255us n=20480 d=0 i=20480 vmx=99.6% 20396

2019-07-25T19:52:05.565-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=1788us pavg=15440us n=22528 d=0 i=22528 vmx=99.6% 22444

2019-07-25T19:52:35.147-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=8352us pavg=80549us n=24576 d=0 i=24576 vmx=99.7% 24492

2019-07-25T19:53:05.444-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=8186us pavg=6193us n=26624 d=0 i=26624 vmx=99.7% 26540

2019-07-25T19:53:33.828-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=11780us pavg=58511us n=28672 d=0 i=28672 vmx=99.7% 28588

2019-07-25T19:53:59.384-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=12933us pavg=29066us n=30720 d=0 i=30720 vmx=99.7% 30636

2019-07-25T19:54:26.636-08:00| vmx| I125: DISK: diskstats for scsi0:0 avg=13732us pavg=25724us n=32768 d=0 i=32768 vmx=99.7% 32684

"pavg" is a periodic average ... the first several thousand I/Os are typical for fast I/O (~0.3-0.5ms) - that's buffer cache or an SSD. Then things slow down dramatically (20-80ms) - note an HDD has I/O latencies of 10ms, and we're on Apple hardware so these are actually SSDs which should be a lot faster. On 10.13, the same workload stays at the low numbers forever.

The reason this impacts encrypted VMs worse is lock contention. The cryptography extends the cost of an I/O somewhat, and it looks like this cost just enough that things go pathological - we're getting 20% lock contention on storage locks. That's the software equivalent of highway traffic ... fast when nobody is on the road, but get enough cars to occupy 20% of all road area and it's rush hour for everyone.

The "accessibility" option pointed out by a few posts is puzzling. That option affects our ability to "warp" the mouse, which you normally see by the mouse being grabbed and not wandering out of the window by accident. But that's on the mouse input path, nowhere near the storage path.

We're still looking into workarounds and solutions.

[ksc]

The one suggestion I can offer so far is "Settings -> Advanced -> Hard disk buffering" to "Disabled". Needs to be set while VM is off or suspended.

Haven't confirmed this one, but with that set to disabled the I/O latency on my test system aren't spiking beyond ~4-6ms, which is tolerable. Still 10x slower than ideal, but not the 100x seen otherwise.

If this helps - or has no effect - would appreciate feedback.

(Buffering helps quite a bunch for the "4Kn" issue. So disabling buffering should make things worse. Unless buffering has gone horribly wrong...)

[prpsarathy]

It is sad that we are not getting response from vmware nor apple.  probably this experience is giving me the thought probably i must move away from expensive and un concerned expense apple eco system

[sfella1005]

I gave it a try and it doesn't seem to help.  Now I have to force quit Fusion so I can get my non-encrypted VM functioning.

[Zongmin]

Hi all,

Firstly, we just want to thank everyone for sharing your experiences on this issue with us and the community, we’re very grateful.

We have been able to reproduce this issue in-house, and we’re actively working on a solution.

So far, we’re seeing 3 common scenarios in this thread:

1. Encrypted VM

2. File Vault being used

3. No encryption involved at all

We have been able to reproduce #1, and the engineering team is currently at work on the others.

Logs and more details from the community is always appreciated and helps even if we can’t reply to every post.

Thanks for working with us on this, it caught us by surprise for sure. Thank you for your patience and we will keep you updated once we have further progress.

VMware Fusion Team

[Sven1802]

Hi, for me it does not have any effect. I'm having a encrypted VMware image and do not use FileVault.

[zhus]

A workaround in my env. FYI.

After changing the memory of the VM (in Settings -> Processors & Memory) from 4G to 2G, it works.

Hardware:

MacBook Pro 13,3  (15inch 2016)

Processor 2.7GHz intel core i7

Memory 16GB

My VM: Windows 10 1809 Enterprise x64 (CPU 2cores, Memory 2GB), Ubuntu 18.04.2(CPU 1core, Memory 2GB)

VM encrypted: Yes

Disk encrypted: No

[reihl]

How can we are export logs for you? Can you please elaborate what do you need and how can we send it to you?

The solution with the accessibility isn't permanent. After rebooting the CPU is sky high 100% and the disk also 100%

[cloud791]

I've just tried with the suggested workaround to Disable and Re-Enable the check on the Privacy Tab of Security Settings, but unfortunately it doesn't change anything...

My Virtual Machines (No Encryption on Guest OS or VM itself, No FileVault on MacOS) running on VMWare Fusion Pro 11.1.0 are still unusable (both Windows 7 and Windows 2012 R2 guest OS). I'm not able to work anymore!! Please, fix it asap!!

Thank you so much.

[unkindone]

My summarized steps for issue resolution:

1. Deleted all snapshots

2. Unchecked split in CM: VM settings --> Hard Drive

3. Unencrypted VM disk: VM settings --> Encryption

     Note. If disk is corrupted, then repair it: sudo /Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -R _YOUR_DISK_IMAGE_NAME.vmdk

4. Checked/unchecked Vmware fusion in macos Settings --> Security --> Privacy -- Accessibility

5. Set "Hard disk buffering" => "Disabled" in VM settings --> Advanced

6. Set "Troubleshooting" => "None" in VM settings --> Advanced

[cloud791]

I've tried your steps. More in detail:

1. Deleted all snapshots   >>>> Never made a Snapshot of my Windows 7 Virtual Machine

2. Unchecked split in CM: VM settings --> Hard Drive  >>>> The Virtual Machine was originally created with one disk file and no splitting option at all

3. Unencrypted VM disk: VM settings --> Encryption  >>>> Never encrypted my Windows 7 Virtual Machine from VM Settings. No encryption software is installed into the Guest Windows 7 VM.

     Note. If disk is corrupted, then repair it: sudo /Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -R _YOUR_DISK_IMAGE_NAME.vmdk

4. Checked/unchecked Vmware fusion in macos Settings --> Security --> Privacy -- Accessibility  >>>> Ok I've successfully executed the sequence steps

5. Set "Hard disk buffering" => "Disabled" in VM settings --> Advanced  >>>> Ok I've changed this setting from Automatic to Disabled

6. Set "Troubleshooting" => "None" in VM settings --> Advanced  >>>> Ok I've changed this setting from Default to None

The result is that, after successfully entering the credentials into Windows 7 Login Screen, it is more than 10 minutes that I'm waiting into a black screen.

Every time I try to move my mouse pointer inside the black screen or to click in order to open any Menu from the VMWare top or side bar, the mouse pointer changes into the rotating rainbow circle icon for around one minute before having control on the menus.. it's horrible, it's more than 2 days that I'm not able to work anymore!!

Please consider that the VMware Tools installed in the Guest Windows 7 Professional machines are the latest version (v. 10.3.10 64-bit).

The problem has started just after the upgrade from MacOS 10.14.5 to MacOS 10.14.6. My MacBook Pro has NO FileVault active and NO File Encryption or AntiVirus programs installed on it.

The SSD is an APFS Volume.

Please fix the problem with Apple ASAP!!!!

Thank you.

[F_Ps]

I tried the setting for Hard Disk buffering (setting it to "Disabled") and it does not make a difference.

[tonyfortpedroco]

Thanks for the official reply. I appreciate the difficulty of fixing things under time pressure when the ground shifts beneath you. Please let us know if there is any additional info we can provide, or troubleshooting we can do to help.

As others have indicated, we (I) rely heavily on VMware Fusion to get work done during the day, and this performance issue is either a showstopper or a major drag on our ability to get things done, until the issue is resolved. I've never downgraded a MacOS X installation, but I wasn't seeing this behavior until I upgraded to 10.14.6 the other day, and I might consider downgrading back to 10.14.5 if the dev team doesn't think a fix will be forthcoming in the next few days.

[chamberl66]

Disabling HD buffering seems to have helped so far.  The Windows 10 VM is definitely usable, unlike before.  Granted, I don't have high performance needs, but I can at least support my customer again.

Thanks,

Dave

[sfella1005]

I tried your work around and it seems to fix my issues.  I had 8gb and changed it to 2gb.  Going to see how it works throughout the day.

**UPDATE**

So far so good as long as the memory is set to 2gb.  Would still love to have move due to some applications I have to run.

**UPDATE 2**

Just ran into this error.  Don't know if this is related or not.

[Jaybeee]

I had same issues and tried to find out how to fix it and read that someone fixed it by disabling encryption so I tried it and it worked for me. All of my VMs used encryption so I unchecked "Enable Encryption" on Encryption & Restrictions of Settings. I hope this will work for you as well.