Announcing - The Unofficial Fusion 13 for Apple Silicon Companion

[Technogeezer]

In the time honored tradition (well, at least for the last year) of the unofficial Tips/Techniques for Fusion Tech Preview documents. I proudly present:

The Unofficial Fusion 13 for Apple Silicon Companion

The document that VMware didn't publish, and you didn't know that you needed. Until you do.

The document may be found at https://communities.vmware.com/t5/VMware-Fusion-Documents/The-Unofficial-Fusion-13-for-Apple-Silicon...

The Companion document is a collection of tips and techniques that were provided by the broader Fusion community while running the Tech Preview releases on Apple Silicon Macs. It can save you searching the forum for frequently encountered issues. 

The Companion may answer many questions that aren't covered by reading the VMware Fusion 13 documentation. Take a look at it before you start creating VMs on Apple Silicon Macs - you might find something that will make your life easier. If you have a question that you're thinking about posting in the forum, check out the Companion first - it may already contain an answer to your question.

Of particular interest to Windows users is the section on Windows 11 on ARM. This section is chock full of information, procedures, and screen shots that should help you be successful in the installation of Windows 11 on ARM on Fusion 13.

For you Linux arm64 users, you haven't been left out. There is plenty of content to help you as well, especially if you use Ubuntu or Fedora.

This document will be updated as the community experience with Fusion 13 grows and is discussed in the forums. 

Enjoy and please post any comments or questions about the Companion to the Fusion forums.

[palter]

[Edited to correct pricing on UTM]

Excellent work as usual!

I'd like to mention another alternative for virtualization of macOS on Apple silicon -- UTM which is only $9.99 and available on the Mac App Store.

The latest version also allows virtualization of Linux with support for Rosetta 2 in Linux. (Unfortunately, Linux VMs are highly unstable on base M1 systems, apparently an Apple and/or Linux problem.) When I finally upgrade my mini, I hope to use that feature to run my Alpha VMs faster than my Intel NUC can today.

[Technogeezer]

Thanks for the reminder on UTM's virtualization capabilities. It was a little off-topic of my original intent for the document, but I'll see if I can find an appropriate home for it in the document since I did mention VirtualBuddy as an alternative for macOS virtualization.

UTM can also be downloaded for free from its website https://mac.getutm.app - but buying through the App Store helps funds future development.

[tallLeRoy]

I made two Windows 11 Pro .iso files from uupdump.net. I used the same source but built one using a Windows VM and the other using the HomeBrew method in MacOS.

The Windows built .iso was complete except for the Microsoft Store. The HomeBrew build was not complete. Microsoft.SecHealthUI was not included in the .iso; and it is not easily available to complete the Security system. 

Bottom line: USE WINDOWS TO BUILD YOUR WINDOWS 11 .ISO.

You could use the incomplete Windows VM from the HomeBrew .iso to build the Windows version of the .iso to get a more complete security system as well as other things not included with the HomeBrew build.

[Technogeezer]

@tallLeRoy - thanks for the update.  Question for you. What build did you use from uupdump.net?

I have a sneaky suspicion that the 22H2 builds might not build as well as the 21H2 versions on macOS. I wonder if the situation is any better if you start with a 21H2 ISO, then let it update to 22H2 via Windows Update?

Additionally,  how did you determine that Microsoft.SecHealthUI was not installed? I'd like to focus some testing on making sure that I can verify how well procedures are working.

I want to confirm this before updating the Companion.

[Technogeezer]

First test on macOS on Apple Silicon.

I selected the latest 21H2 build 22000.1281 (labelled as an Insider Preview channel release) in uupdump and de-selected apply updates from the final page (because it won't be done anyway since I'm building on macOS). Then ran the macOS scripts.

The resulting ISO was build 22000.1. I used that to install Windows 11 Pro in a fresh VM. I used the instructions in the Companion to install from the ISO (inserting the drivers at the point described in the Companion).

The system updated itself during install to 22000.318. I did run Windows Update after that and a few more updates installed. Not unexpected.

After install, the Microsoft Store was visible in the system tray, but wouldn't start. I dropped into a command prompt as administrator and used "wsreset.exe -i' to fully install the Microsoft Store. After install of Store Experience Host and Microsoft Store, the store would now execute.

I did not have any issue with starting Windows Security from the Privacy & Security section of Settings.

The VM was updated via Windows Update to Windows 11 21H2 22000.1281. It then offered to update itself to Windows 11 22H2 (as expected). The update finished with 22H2 22621.819 installed. Microsoft Store and Windows Security works.

So far so good.

Now on to trying installation from 22H2 (22621.xxx) built on macOS...

[Technogeezer]

Second test by building the ISO from uudump on macOS

I selected the Latest Public Release build from uupdump - 22621.819. Like the last test, on the final page, I deselected apply updates.

Then built the ISO on macOS. The resulting build was 22621.1. Like the prior test, I installed using the procedure in the Companion doc.

After install, winver reported Windows 22H2 22621.1

Performed Windows Update. Offered to update Security Intelligence, .NET framework, and the 2022-11 Cumulative Update for Windows 11 22H2, and I updated all 3. Rebooted to complete installation of the updates.

Winver now reports 22621.819. A second check for updates after installation reports that there is an update for Microsoft Defender.

Windows Store not present, so I dropped into a command prompt (run as administrator), and executed wsreset.exe -i. As before, the Store Experience Host and Microsoft Store downloaded and installed.

At this point, Windows Security would not open - said it needed another app.

I re-checked Windows Update, and it said that there is an.update for the Microsoft Defender platform. I installed it. Still says that I need another app to open Windows Security.

It appears too complex of a process to install from the 22H2 Windows 11 builds from uupdump. I'm going to reinstate in the Companion my prior recommendation (that I had in the Tech Preview Tips and Techniques guide to build the 21H2 release from uupdum. Then it will update to 22H2 and all should be working well.

I do thank you for catching this, and I apologize to the community for not testing this out as thoroughly as I should have before including it in the Companion. It will be fixed and a new version released as soon as possible.

[Technogeezer]

An update to the community:

Version 2 of the Companion has just been published. It can be found using the link in the first post of this thread.

Thanks go out to @tallLeRoy for pointing out a serious issue with the document's procedures for building ISO media for Windows 11 on macOS. This version corrects those issues.

------

Updated content in Version 2:

URGENT: Version 2 of the Companion has been updated because of a significant issue discovered with Windows 11 ISO media built on macOS using the procedures in Version 1.. Guidance and procedures on how to successfully build ISO installation media for Windows has changed in Version 2 based on these discoveries. 

Users wishing to use Windows 11 are strongly urged to use the procedures in Version 2 of the Companion (and later), especially if you are going to build ISO installation media on macOS.

A glossary of terms that may be unfamiliar to some users has been included in Version 2.

------

(with apologies to The Treasure of The Sierra Madre)

Videos? We ain't got no videos. We don't need no videos! I don't have to show you any stinkin' videos!

[tallLeRoy]

Thanks @Technogeezer ,

Your response in v2 of the Unofficial guide seems to be right on. 

You might mention that you can do the Windows build on the 22H2 MacOS install then install the Windows build and delete the MacOs built VM. Time drain, but may be shorter than waiting for Microsoft to update from 21H2 to 22H2.

[Technogeezer]

I thought about that. It's certainly an option since the 22H2 ISO will build the "correct" ISO even on the broken Windows VM. I just wanted to keep the procedure for installation as simple as possible. I'll make a note in the next version.

Sometimes, though, less is more. People have a tough enough time with this process, and even something you and I might find simple can cause other peoples' heads to spin.

In my haste, I forgot to acknowledge you in the V2 document. That's fixed and you will be properly cited in the next version. 

Thanks!

[treee]

I have some corrections for the document:

In the acknowledgements at page 1 it is says "running multiple hypervisors". On page 10 the title for 2.6 also mentions "multiple hypervisors" as well as the link that is mentioned there. I would refrain from using that terminology as it is no longer accurate. Due to recent changes in macOS the only hypervisor you can only (or have to) use the in-built one that is generally known as hypervisor.framework. It is best to use more general wording like "virtualization product". This is what is used in the body of chapter 2.6. Perhaps expanding chapter 2.6 with this info would also suffice.

Then there is also the definition of "hypervisor" on page 4. Not sure if that is entirely correct as in general the definition only mentions the creation and running of virtual machines. Lots of hypervisors have outsourced the management part to a different type of tooling (sometimes an app, sometimes a management server). As explained above it remains to be seen whether the mentioned examples are using their own hypervisor or that of macOS itself. They are no longer good examples to use, I feel that ESXi would be a better one.

In chapter 2.1 on page 6 you should also mention that running (docker) containers with vctl is (currently) not possible. The vctl tool is indeed part of the package but it is x86_64 and as a result runs via Rosetta2. When you want to use containers you have to use vctl to start a linux vm with a container runtime. This seems to not work because vctl runs via Rosetta2 and will try to fire up an x86_64 vm. Rosetta 2 does not support virtual machines so starting a vm won't work. Trying to run containers with vctl will result in an error that the vm isn't started. The Fusion manual does mention vctl being Intel-only, you can only find that in the title.

Chapter  2.1 also mentions nested virtualization. The reasoning why that isn't supported is described as if it were facts while in reality they are no more than speculation. There is no official word that nested virtualization isn't supported, let alone that the hardware and software doesn't allow it. We only have some pieces of the puzzle but not the entire puzzle. I think it is best to remove it since the other bullet points also have no explanation as to why they aren't working on Apple Silicon; it is something that simply doesn't matter.

In the next chapter (2.2) it is mentioned that VirtualBox doesn't run on Apple Silicon. This is outdated info as VirtualBox now has a developer preview that runs on Apple Silicon. Could you update that part?

The next chapter is 2.3 after which chapter 2.4 follows except that it doesn't 🙂 The numbering goes back to 2.2 (see page 8). That needs fixing.

Speaking of the second chapter 2.2 (the one on page 8): can that first sentence on page 5 for the definition of open-vm-tools also be used in chapter on page? It is a more accurate description of what open-vm-tools is than what is currently on page 8 and brings that chapter in line with the definition of open-vm-tools.

I also noticed that some distributions have shared issues such as having to blacklist vmwgfx. Would it be an idea to generalise the Linux part of the document and only add a few chapters mentioning specific issues of a Linux distro (an example would be the page size used by RHEL 8.x which is also used by all distros based on RHEL)?

Lastly kudos for noting that OpenSUSE Tumbleweed still has to be created the custom way. The bug I mentioned during the tech preview is still there: when using the easy setup wizard it will use the suse template but unfortunately that template is still x86_64 (this is not a biggy, I think the lack of vctl is a bigger issue). And of course big kudos for making the document in the first place 👏

[Technogeezer]

---

@treee wrote:

I have some corrections for the document:

In the acknowledgements at page 1 it is says "running multiple hypervisors". On page 10 the title for 2.6 also mentions "multiple hypervisors" as well as the link that is mentioned there. I would refrain from using that terminology as it is no longer accurate. Due to recent changes in macOS the only hypervisor you can only (or have to) use the in-built one that is generally known as hypervisor.framework. It is best to use more general wording like "virtualization product". This is what is used in the body of chapter 2.6. Perhaps expanding chapter 2.6 with this info would also suffice.

---

I agree with you on this. Done.

Then there is also the definition of "hypervisor" on page 4. Not sure if that is entirely correct as in general the definition only mentions the creation and running of virtual machines. Lots of hypervisors have outsourced the management part to a different type of tooling (sometimes an app, sometimes a management server). As explained above it remains to be seen whether the mentioned examples are using their own hypervisor or that of macOS itself. They are no longer good examples to use, I feel that ESXi would be a better one.

It's still software that's partitioning and abstracting hardware resources to run multiple operating systems, whether it's implemented by the software itself or outsourced to APIs such as Hyper-V. I do agree that the definition may need to be clarified. 

Chapter  2.1 also mentions nested virtualization. ...   I think it is best to remove it since the other bullet points also have no explanation as to why they aren't working on Apple Silicon; it is something that simply doesn't matter.

Agreed. 

In the next chapter (2.2) it is mentioned that VirtualBox doesn't run on Apple Silicon. This is outdated info as VirtualBox now has a developer preview that runs on Apple Silicon. Could you update that part?

I had not heard that the developer preview of Virtual Box was available for Apple Silicon. Thanks for straightening me out on that!

The next chapter is 2.3 after which chapter 2.4 follows except that it doesn't 🙂 The numbering goes back to 2.2 (see page 8). That needs fixing.

Oops. That's what happens when you proofread your own documents.

Speaking of the second chapter 2.2 (the one on page 8): can that first sentence on page 5 for the definition of open-vm-tools also be used in chapter on page? It is a more accurate description of what open-vm-tools is than what is currently on page 8 and brings that chapter in line with the definition of open-vm-tools.

Not a bad idea. Consistency does count.

I also noticed that some distributions have shared issues such as having to blacklist vmwgfx. Would it be an idea to generalise the Linux part of the document and only add a few chapters mentioning specific issues of a Linux distro (an example would be the page size used by RHEL 8.x which is also used by all distros based on RHEL)? 

I struggled with this one. Let me think about it some more as you have a good point.

Lastly kudos for noting that OpenSUSE Tumbleweed still has to be created the custom way. The bug I mentioned during the tech preview is still there: when using the easy setup wizard it will use the suse template but unfortunately that template is still x86_64.

Now that you mention it. I should make the point a bit clearer that the drag/drop method won't work for Tumbleweed as well as Leap.

Thanks for the feedback. The goal is to make the document as useful to Fusion users as possible, and you've certainly helped.  

[Technogeezer]

Hello and welcome to December 2022. It's time for another update to the Companion. Version 3 can be found in the link posted at the start of this thread.

------

Updated content in Version 3:

The wording of some content has been modified from community feedback to improve its accuracy.

Creation of a new Archived Topics section to preserve workarounds for issues that have been fixed by newer releases of Fusion and/or Linux/Windows versions.

Updating of the RHEL section to reflect the release of RHEL 9.1 and fixing of several issues.

Addition of recommendations for the creation of an OpenSUSE Tumbleweed virtual machine.

Addition of URLs for download of Ubuntu daily releases, some of which were formerly found in the Tech Preview Testing Guide.

Better quality screen shot images in the procedure for building Windows 11 ISO images on the Mac.

Additional items in the glossary.

------

[slither2007]

Thanks for the guide!

I've been running Fedora 37 without issues on my MBP 14. I have noticed that despite 3d acceleration being enabled in its fusion vm settings, I can't actually see Fedora confirming acceleration is enabled. Is this normal?

The experience is still smooth and fluid, window auto-resizing and retina resolution support works great, just thought I'd share this 🙂

The output of glxinfo shows it's not accelerated and misreport the dedicated 8GB vram.

name of display: :0
display: :0 screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
Vendor: VMware, Inc. (0x15ad)
Device: SVGA3D; build: RELEASE; LLVM; (0x406)
Version: 22.2.3
Accelerated: no
Video memory: 1MB
Unified memory: no
Preferred profile: core (0x1)
Max core profile version: 4.3
Max compat profile version: 4.3
Max GLES1 profile version: 1.1
Max GLES[23] profile version: 3.1
OpenGL vendor string: VMware, Inc.
OpenGL renderer string: SVGA3D; build: RELEASE; LLVM;
OpenGL core profile version string: 4.3 (Core Profile) Mesa 22.2.3
OpenGL core profile shading language version string: 4.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.3 (Compatibility Profile) Mesa 22.2.3
OpenGL shading language version string: 4.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 22.2.3
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

I also see this in Chrome when checking gpu support in chrome://gpu

Graphics Feature Status
Canvas: Software only, hardware acceleration unavailable
Canvas out-of-process rasterization: Disabled
Direct Rendering Display Compositor: Disabled
Compositing: Software only. Hardware acceleration disabled
Multiple Raster Threads: Enabled
OpenGL: Disabled
Rasterization: Software only. Hardware acceleration disabled
Raw Draw: Disabled
Video Decode: Software only. Hardware acceleration disabled
Video Encode: Software only. Hardware acceleration disabled
Vulkan: Disabled
WebGL: Software only, hardware acceleration unavailable
WebGL2: Software only, hardware acceleration unavailable
WebGPU: Disabled
Problems Detected
WebGPU has been disabled via blocklist or the command line.
Disabled Features: webgpu
Accelerated video encode has been disabled, either via blocklist, about:flags or the command line.
Disabled Features: video_encode
Accelerated video decode has been disabled, either via blocklist, about:flags or the command line.
Disabled Features: video_decode
Gpu compositing has been disabled, either via blocklist, about:flags or the command line. The browser will fall back to software compositing and hardware acceleration will be unavailable.
Disabled Features: gpu_compositing
...

[Technogeezer]

What you are seeing is normal (from what I see) from glxinfo when you enable 3D acceleration for the VM. The extended rendering info is your key - denoting the detection of the VMware SVGA3D device. That device is not detected if you turn off 3D acceleration for the VM.

Also, from what I read on the web, Chrome on Linux is a particularly bad program to judge whether GPU support is enabled in Linux or not. There are articles as recent as earlier this year saying that Google has not and will not enable hardware GPU acceleration for Chrome on Linux. Even if you think you've turned it on within Chrome's preferences, there's still no GPU acceleration.

The excuses given by Google are that 1) Google values stability of the browser over GPU acceleration, 2) they don't have the resources to do it, and 3) that the Linux GPU 3D device drivers are immature. Pick the excuse you wish, but it looks like they all lead to the same destination. No GPU acceleration for you, Linux.

There are supposedly some unofficial patches out there for Chrome that some say work, and others say don't. 

If you want to see if 3D is really there, I like running two programs: glxgears and glmark2.

glxgears will seem to give higher frame rates when when 3D support is disabled for the VM, but you will see that lots of CPU time is being used and the frame rate will drop as you increase the window size.  When 3D support is enabled, glxgears ties its frame rate to the refresh rate of the screen (typically 60Hz) making the frame rate lower than without 3D support. But glxgears uses very little CPU time in the VM, and the frame rate stays pretty constant if you resize its window.

On my Mac mini M1, glmark2 gives a composite score of 350 with 65-80% cpu time when 3D support is disabled. Enable 3D support, and the composite score rises to 2900 with < 15% cpu time in the VM.

[Technogeezer]

The Companion has been updated to version 4. It can be found via that link at in the first post in this thread.

This is a minor update that corrects the links for download of Ubuntu installation ISOs, and adds information on the Ubuntu 23.04 development daily builds. 

[Technogeezer]

Happy New Year to everyone (albeit a couple of days early).

The Companion has been updated to version 5. It can still be found via the link in the first post in this thread.

Updated content in Version 5:

Details on the requirements for 3D support on Linux guests has been added, filling in the missing pieces that VMware did not include in the release notes or documentation. Several questions about this have been asked recently in the forums.

The section on the 'Start pxe over IPv4" message has been moved to the General Notes section and enhanced, as it impacts all guests.

Multiple monitors and Unity have been added to the list of features not supported by Fusion 13 on Apple Silicon.

Text mode installation now works for latest versions of the CentOS 9 Stream installer. The former workaround has been moved to the Archived Topics section.

The Glossary has been moved to the end of the document.

Minor formatting issues such as typos, section numbering, and table layout have been corrected.

Request to the community for any info on what works and what doesn't in the Oracle VirtualBox 7 developer preview on Apple Silicon - what really works and what doesn't because it's not clear from any documentation provided by VirtualBox.

------

[Technogeezer]

Version 6 of the Companion has just been published. 

First update of 2023, and an important one if you use OpenSUSE Tumbleweed! See the first link of this thread, or search the forum for the term "Companion".

Updated content in Version 6:

A workaround is included for a bug recently found in OpenSUSE Tumbleweed snapshots 20221219 and later. The bug prevents installation from newly download installer ISOs, and breaks existing VMs updating to the latest Tumbleweed snapshot. This bug is not exclusive to Fusion on Apple Silicon. It impacts any physical and virtual ARM64 architecture installation.

Fixed some typos in the Ubuntu 22.04.1 download links. The links work properly but the descriptive text in the table had some fat-fingered version numbers. 

[Technogeezer]

Version 7 of the Companion Guide has just been published.  See the first post of this thread for the link to the guide, or search the forum for the term "Companion".

Tumbleweed and Windows users will be happy with this update to the guide. Ubuntu users, well, not so much thanks to Ubuntu.

Updated content in Version 7:

• Addition of a "known good distribution" list earlier in the guide. That may help users new to Fusion 13 to select a distribution.
• Update to the OpenSUSE Tumbleweed installation and upgrade issues noted in version 6 of the guide (short version: they're fixed!).
• Re-verification of Ubuntu download links. Bad news, though. Ubuntu has removed arm64 ISOs for the pre-release 23.04 daily builds. 😠
• Clarification on how to obtain Ubuntu Desktop on ARM (short version: they don't make it easy).😠
• Addition of info on Microsoft licensing and activation for Windows 11 ARM.
• Addition of information to the Windows 11 installation from ISO procedure. The procedure has been enhanced with screen shots of of Windows Setup immediately after boot from an installation ISO.
• Update of free disk space requirements when building ISOs from uupdump.net on Windows.

[Technogeezer]

Version 8 of the Companion Guide is now available. See the first post of this thread for the link to the guide, or search the forum for the term "Companion".

This update contains content will be of interest to those that use MacPorts instead of Homebrew for open source package management.

Changed in version 8:

• Added of OpenSUSE Leap 15.5 alpha as a distro known to work with Fusion.
• Re-branded the Glossary as Appendix A
• Moved instructions for obtaining open source utilities from Homebrew into Appendix B. 
• Added an Appendix C containing instructions for obtaining open source utilities from MacPorts as an alternative to Homebrew.
• Procedures in the Windows 11 ARM section remove references to Homebrew. They now assume that required open source utilities have been installed from either Appendix B (Homebrew) or Appendix C (MacPorts).