I am not familiar with TMG Forefont.
What is the configuration of your Mac host? What about the configuration of the Windows 2008 R2 VM? Does it also meet the recommendations for TMG Forefront?
The recommendation seems to be at a minimum Core 2 Duo (Penryn) or the Nehalem CPU. Unless you using a very old Mac hardware to run the VM, most likely it should be meet the CPU requirement although CPU features are not necessarily available in all models of the same generation (e.g. a Nehalem Xeon CPU might have features that a laptop/desktop Nehalem generation CPU does not).
Alternatively, you can check if the hardware compatibility version of the VM is up to the version that your Fusion can support (e.g. Fusion 8.x supports v12, Fusion 10.x supports v14). Using an older/earlier hardware compatibility version can mask CPU features from the VM. I think the guest OS setting might also affect that as well.
I'm running VM Fusion Professional 10.1.1 on Macbook Pro OX X Sierra 10.12.6
Hardware on VMWare Fusion Comparability is "14"
2 Processors, 2 Gig RAM configured in Fusion for Server, no Domain.... Just fresh install from image. and attempting to install TMG forefont. on x64 bit OS
On MS site i can only find x32 on TMG 2010, as i'm after this version specially. So it should work, as you can install 32-bit programs on 64-bit OS.
I haven't run Windows Update yet. I know I haven't installed .NET or anything per-requisite based on the Deployment Guide, But installation should able to start.
In general, you could run install and run 32-bit Windows applications on an 64-bit Windows OS. However, it looks like TM Forefront looks like has a dependency of IIS. The terms "Secure Web Gateway", "Secure Mail Gateway", "Proxy Server", HTTPS inspection would suggest this product also has a dependency on IIS Web Services. So if IIS web services (possibly other services such as SMTP, etc) has to be installed even though it is not looks like to be explicitly stated. So that could be one of the problems you have face.
I don't know how far you can go with running 32-bit components on 64-bit IIS server.
In addition to that, anything with .NET Framework dependency, generally, the corresponding .NET Framework version has to be installed first. The registration of COM/COM+ components that come with any installation will also fail without the proper .NET Framework version installed first.
It looks like the problem you have does not have anything to do with Fusion.