Trying to get pass through working on a standalone workstation.

[Urumiko]

So I've upgraded my PC for the first time in about 8 years and I thought i'd be bold and have a go at running my primary desktop environment (windows) as a VM with Graphics, USB etc passed through. allowing me to run my primary desktop, my NAS, and any lab machines in one box.

My motherboard is a Asrock x570 Phantom gaming 4.

I've gone with a free vsphere 7 licence in the first instance.

So far I have figured out how to enable both SR-IOV. and IOMMU in the bios. (which was not easy, damn how many options do BIOS have nowadays)

And I'ive actually managed to pass through my NVME drive and install windows to it.

Under hardware > pci devices

I note that despite enabling SR-IOV in the BIOS everything says Not capable

The web gui is extremely buggy. If i try and select my GPU to pass through, it seems to try and auto select the 3 associated components of this device but they do an erratic dance where it rapidly selects and deselects indefinitely until i refresh the page. (I've tried 3 browsers)

Yesterday I managed to get to the stage where I had the GPU passed through.

I just needed to pass the USB3 hubs through so i could use my mouse and keyboard.

It was continually saying that pass through would be enabled on restart but when i restarted it still said that.

On the rare occasion I could get a USB hub passed through the VM would not start with the error:

"Failed to register the device pciPassthru1 for 10:0.3 due to unavailable hardware or software support."

I tried updating to the latest bios to the latest revision which lists:

Update AMD AGESA Combo-AM4 V2 1.0.0.2

This did seem to result in the PCI IDs swapping round a bit and it broke my GPU pass through but no progress.

So im a bit stuck.

Is there anything further I can try?

I know these techs can require further manual config but I'm completely new to this so I don't know where to start.

Would there be any benefit to trying ESX6.5, or perhaps proxmox?

Thanks.

[scott28tt]

Info on your hardware would make this a lot easier:

System make/model

IO devices

GPU

etc

Sounds like you might be using items which aren’t supported, which may or may not contribute to your issues...

[daphnissov]

(Sigh...)

I really wish people would stop trying to do stuff like this. First of all, everything that you have and what you're doing is unsupported, so you're guessing as to if it will work at all. Second, if you want to use virtualization but bastardize it by passing through loads of physical hardware, just use physical hardware in the first place. Virtualization is about abstracting away physical hardware in order to present common, standardized sets of virtual hardware that a guest OS sees. But you're throwing all that in the garbage. Pass-through was only ever designed for certain corner cases and not for generic use. So if what you want is pass-through, don't use virtualization. Your life will be much easier.

[Urumiko]

Oh absolutely not supported but I figured I'd give it a fair crack of the whip in the name of exploration.

Motherboard: ASRock X570 phantom gaming 4., rear USB is 3.2 gen 1, and 3.2 gen 2, I am unsure of exact controllers.

GPU: Nvidia GTX 1660

CPU is Ryzen 3 3100 though i intend to upgrade it soon.

storage is a combination of NVME (PCI3), and sata.

[Urumiko]

I do get where you are coming from but there are a growing number of enthusiasts with high core and ram counts who see value in ruining multiple OS's on one box.

Of course that's not vmwares focus, and there are others out there like Unraid more focused on this but I thought id try the more robust offerings first.

Yes there is vmware workstation of course but that's not really suited to discrete always on instances.

And that does have one  major drawback in that i cannot pass through my ZFS pool to it.

If you can think of a better approach to allow me to have a windows workstation and a ZFS fileserver running on one box im all ears.

[daphnissov]

It's always been possible to run multiple operating systems on a single box, but what doesn't work well is when you take a single physical box, layer in virtualization to abstract hardware away, then loop that same hardware that you're trying to abstract back in. This is one of the most common complaints that arise on these forums:  "help me, I'm trying to pass through my <insert_consumer_hardware_here> to a <insert_desktop_OS_here> and it's broken!" And the reason is that, number one, this pattern is generally anti-virtualization, and number two, ESXi is not designed for consumer hardware. When you combine both things you have a very precarious combination that either doesn't work well, doesn't work at all, or produces instability down the line.

Secondarily, you're using a z-pool which means you've probably using it for some sort of persistent storage. On no account would I ever want that concern lumped in with a gaming concern. At least, not if you value what you're storing. I would separate that into its own physical box and use whatever you want (FreeNAS, whatever) to present as a NAS. On your main desktop system, just leave that physical.

[sjesse]

Vmware workstation is fine for leaving it one, the only downside is if your using windows or a host os that auto restarts it restarts your vms. I've done it for year and only recently got a dedicated esxi server. Why don' you just run freenas without pass through, any non persistent use case isn't going to notice. I'm running 16 servers off of one virtualized freenas instance, which includes testing virtual desktops which can be io instensive.You can also get some cheap servers on line that can't run esxi anymore that can freenas just fine for under $300 USD.