Linux Guest OS - OpenCL support (OpenCL over Vulkan over MoltenVK over Metal wrapper?)

Linux Guest OS - OpenCL support (OpenCL over Vulkan over MoltenVK over Metal wrapper?)

Status: New Submitted by Contributor on ‎02-10-2022 08:02 AM

It seems like it will be a lot of translation laters -- but getting OpenCL support in Linux will be a game changer -- the "easiest" way I could think is OCL on Vulkan. But then will need Vulkan on MoltenVK on Metal so things could easily break. That being said, this is proprietary software, so that should be fine.

3 Comments
Eirikr1848
Contributor
Contributor
‎02-10-2022 08:03 AM
‎02-10-2022 08:03 AM

Whoops, can't edit that. Definitely meant "translation layers" not "laters" :slightly_smiling_face:

Mikero
Community Manager
Community Manager
‎02-10-2022 02:13 PM
‎02-10-2022 02:13 PM

Thanks for sharing :grinning_face_with_big_eyes:

What would be the use case for this? 

Eirikr1848
Contributor
Contributor
‎02-11-2022 02:28 AM
‎02-11-2022 02:28 AM

With Apple having deprecated OpenCL, its only a matter of time before they remove the ability to use it from the OS.

So the first use case is VMWare providing OpenCL to Mac users who will soon not have it. As far as each person's use case goes... that could be anything from Blender to GEGL for OpenCL on GIMP.

Could also be used for any sort of open source video editor with x264 and x265 for example being OpenCL enhanced in terms of performance. FFMPEG and Handbrake would benefit from transcoding as well.

So: essentially, for those who utilize open-sourced applications who currently benefit from OpenCL acceleration on multiple platforms, will lose that acceleration eventually. However, with the M1/Pro/Max all having the tensor cores "neural engine cores" and 7+ GPU cores, in addition to the CPU cores it seems there is a lot of compute power just sitting there.

-----

So I guess:

  1. VMWare Desktop Hypervisor = ARM Native / Written in Swift (and Rust as needed) - Runs Entirely on Metal.
  2. Interfaces with vDSP API for the tensor/neural cores to enhance multimedia processing/rendering/speech synthesis/audio-visual software, DJ apps, Xray and MRI Linux apps for doctors, etc.
  3. open-vm-tools / VMWare Tools could then be used to "hook" into the Metal API easily due to the entire application being metal-native.
  4. Therefore, the OpenGL drivers would just need OpenGL --> Vulkan --> MoltenVK --> Metal translation
  5. The "zink" mesa driver exists already for OpenGL on Vulkan.
  6. So, zink could provide OpenGL via vulkan being installed in the VM.
  7. MoltenVK is baked into the Desktop Hypervisor, so it handles translating the zinkOGL<->Vulkan 3D rendering driver to Metal for near-native performance (after optimizations over time) 
  8. Regarding OpenCL: it could have two branches: One for OpenCL --> Vulkan --> MVK/Metal | OpenCL --> vDSP API --> 16 tensor/neural cores.
  9. Using LLVM/SPIRV, etc (my knowledge is limited) - the OpenCL ICD could then "combine" those two branches to automatically distribute the compute load based on "type" of load (possibly -- or just have two OpenCL devices. Though combined is better of course)
  10. All these "close-to-metal" performance via Linux means that WINE would also be able to be installed for Windows gaming on Linux on Mac ARM (since native ARM Windows is still not permitted). 
  11. The "TURNIP" driver team has been working on porting more x86_64 Windows games to ARM Linux 
Apple Developer website linked in "item 2" wrote:

The vDSP API provides mathematical functions for applications such as speech, sound, audio, and video processing, diagnostic medical imaging, radar signal processing, seismic analysis, and scientific data processing.

So in conclusion --- a complete overhaul of the application in Swift/Metal would result in the top platform for near-native Linux performance without having to mess around with dual booting and reverse engineering drivers. This would be extremely useful for:

  • the medical field and medical research fields
  • weather stations and research stations
  • astronomy research stations
  • DJs who use open source software
  • Content creators who use open source software
  • Windows Gamers who prefer to use their Mac, but get an itch to play a game like (for example) Overwatch that isn't available on OS X but will work with Lutris/Proton/WINE.

The groundwork and framework has been laid by the world's leading virtualization experts (VMWare) to make this happen.

Interface as much as possible with the Open Source community for drivers, "translation layers", "API Hook layers", etc etc as possible to reduce strain on internal resources.

(The core of the program could be proprietary, but with open source code used to make everything work just fine and dandy!)

-----

  1. The lessons learned from this could apply to VMWare on Windows-ARM devices, Linux on native ARM devices (PineBook Pro, Raspberry Pi for starters) but eventually on higher-end ARM Linux devices as they emerge.
  2. These lessons could also make porting to Linux for other platforms like RISC-V, OpenPOWER, etc a breeze!
  3. The Raptor Talos POWER9 systems for example are really tempting. However, I need a trusted level 2 hypervisor on my system that will offer the performance and stability I need.

---

(Eventually, you could be able to emulate other architectures so POWER9 could run ARM Linux and x86 Windows, for example, or ARMv8 could run x86 Windows and RISC-V native, etc etc)

----------------------

Clearly by no means am I a developer! I am an enthusiast, but have been brainstorming today after you asked me about potential use cases. I hope this all makes sense because as mentioned; was just a brainstorm session :slightly_smiling_face:

However, I think there are many opportunities to sell this revised platform not just to Mac users, but to make more as mentioned -- and the lessons learned can help the enterprise ESX side as well!

------