any luck with the apple update? is it improving anything?
I was having the symptoms (periodic freezing) after installing 10.14.6. The only thing that eliminated the symptoms for me was reducing the Win10 guest to 2GB of memory.
Installed the OSX 10.14.6 Supplemental Update just now and restarted the Win10 guest after bumping its config back up to 4GB memory, and symptoms returned.
Reduced the Win10 guest back to 2GB and restarted and symptoms are gone again.
I don't think the supplemental update (all 950MB of it, which is clearly doing something more than correcting an issue where certain models have trouble sleeping) did anything to alleviate this issue we're having with VMWare Fusion.
After a while and several changes to the vm config... nothing really changed with it. sigh.
Getting super old.
7 people found this helpful
Official update... root cause identified and reduced to a small sample program. Still investigating workarounds; until we ship an update, smaller VM sizes or decrypting is going to be the best option. No ETA yet.
(I just reported this root cause to Apple about 15 minutes ago, so I'm almost certain it's not related to their supplemental update.)
The root cause, if curious, is the behavior of paging memory from an unlinked file. Posix reference counts file handles independently from the on-disk entry, so it's possible to create and open a file, make it a certain size, unlink it, then mmap the file handle to a chunk of memory. The operating system will still page to and from the file; when the file handle is closed, the operating system will release the disk blocks too. It's an old-school UNIX trick to get shared memory that automatically gets cleaned up after the last process using it exits. On 10.14.6, accesses to the mmapped parts of the file get slower ... and ... slower ... over time. If the exact same file is not unlinked, accesses slow down some but are still very usable.
Fusion uses this unlinked-file technique for encrypted VMs. (Normal VMs are memory-mapped to the .vmem file, but that obviously doesn't work when the file is encrypted).
Long-time VMware users who have used Workstation may remember the "mainMem.useNamedFile" option. On Linux and Windows, we had the opposite problem: a memory-mapped file was slow, but switching to an unlinked file (sometimes) made things much faster, especially when Linux re-tuned the memory manager. This is the first time we've seen MacOS re-tune the memory manager, and having the first time be in a dot release was quite a surprise. Alas, that option won't do any good here: encrypted VMs cannot use a named file so ignore it.
Guys, hurry up, please! Other "Virtualisators" work flawlessly under 10.14.6 or Catalina.
A machine equipped with only 2 GB RAM isn't the perfect solution under some
thanks for the detailed answer. Just to be sure I've tested the supplemental update. It did not change anything to the slow down issue with VMware.
How does this affect unencrypted guests? I have problems wirh macOS and Windows guests since the 10.14.6 upgrade. I can sort of work for awhile by dropping memory and cores. For example 16GB 6 cores MacBook Pro 15" 2018 dropping 8GB to 6GB for VMs works with a drop of 2 to 1 cores.
Same here, also posted my config a few pages back.
FileVault is on, VM is not encrypted.
Still having problems, also sluggish screen when changing fullscreen desktops between VM and OSX.
I have exactly the same issue on a mid-2017 13 inch Macbook Pro with 16G of memory and not encrypted VM that runs the bootcamp partition. VM is unusable now and I am just running Windows by launching Bootcamp. I have tried to reduce the 8GB allocated memory, which used to work well, to 2Gb. It makes my machine unusable because the software I run requires the additional memory. Please issue a fix asap.
8 people found this helpful
Another official update...
Apple has confirmed reproduction and identified the bug on their end. No timeline for a fix shared yet.
In light of that, we're also working on a workaround. "Days but not weeks", can't be more specific because the workaround is not yet written.
A few more things we know now:
- Affects hosts with >=4GB and <32GB of memory. (Yes, really).
- Triggers after 2GB of modified data is cached, regardless of file. Once triggered, it stays triggered as long as the file is open.
- So why does this affect encrypted VMs more than unencrypted?
- Due to different code paths, this cache bug only affects emulated device I/O. The virtual CPU is not affected.
- Encrypted VMs are much more sensitive to additional I/O costs, and "touch" memory many extra times (due to crypto internals).
- Some benchmarking I did suggests we get ~4MBps touching memory backed by an affected file in normal usage, but crypto's high-touch access pattern drops this to ~0.2MBps or worse. In comparison, an SSD on random-read workloads is 75-100MBps, a HDD is 1-2MBps. So an unencrypted VM "feels" like HDD speed, and an encrypted VM "feels" 10x worse than an HDD.
- In theory, this should affect everybody who is writing >2GB to a file. In practice, most applications writing that much data to a file do so sequentially, which makes it easy to write out modified data fast enough that the bug never triggers. Virtual machines ... have to emulate RAM, which has a very different access pattern.
Really gory details, for the technically inclined ... we know where the actual bug is. This is a link to an (older, 10.14.1) Apple source code drop.
This is a function that keeps a hash table of blocks used from a file; the hash table automatically grows once it reaches a certain size. Apple appears to have "fixed" the XXX comment above the function and added a 32GB case (in addition to the 4GB case already present) ... but their new case has a bug that re-allocates the entire hash table every time.
Once they release sources, I expect the new fix will be a 1-liner to add a return statement in their new code, just like line 6643. Oops.
Hey ksc, really appreciate your updates and full explanations. And I know I can’t be the only one on here that really appreciates them. Outstanding fanatical support and communication Of what’s going on. Thank you.
Thanks for the quick effort and for your transparency around the issue.
I'm impacted by the issue because I use an encrypted VM. I use an encrypted VM because encryption is required when using an emulated TPM. I use an emulated TPM because I have a specific work-related scenario which requires one.
I understand the tradeoffs around a virtualized TPM. What I don't understand is why I'm required to encrypt my virtual disk in order to use a virtual TPM. I understand the need for TPM contents to be encrypted, and it makes sense that you force this. But why do you force encryption of the disk? Would you consider making disk encryption optional when a TPM is in use?
I'll add that, in my case, my virtual disk sits on a FileVault encrypted volume and the machine itself has Apple's access controls enabled. If someone gains physical control of my machine and manages to access the host file system (in order to get at the virtual disk), the situation is already bad and gaining access to the virtual disk won't make it measurably worse. So in my case, I don't consider the second layer of virtual disk encryption to be helpful and I would strongly prefer the performance and battery-life benefits of not encrypting the volume.
I doubt that this is the "workaround" you're looking for in the near term, but the ask here is to remove the requirement for volume encryption when using a TPM. Perhaps put up a warning dialog that forces me to click "OK" to a prompt which reads "I understand that I have chosen to add a virtual TPM without encrypting my drive and I understand and accept responsibility for the consequences of this decision"
First, thanks for the transparent update - that's very much appreciated, and kudos to you and the team for tracking it down.
One question, if we have a 32GB machine, and a VM that only uses 4 or 8GB, why does it open a swap file at all? Or am I misunderstanding?
Aside: I have a sinking feeling that we'll only see a fix in Catalina....
I hope your "sinking feeling" about no fix until Catalina is wrong! I need this fix asap, as we all do.
Thanks for official updates.
No doubt that Apple’s software team is going worse day by day,
but still, VMware didn’t realize this issue during the developer beta is also unbelievable.....
It not only happens in 10.4.6, but also in 10.5 betas!
makes me wonder is fusion still developing or not?
On the other hand, Parallels and Virtual Box don’t have such fatal issue....
I downloaded PD, and trying it for almost 2 week but here the problem still not solved....
Look like I have to buy a license otherwise I cannot work....