ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

radixbl — Wed, 19 Apr 2023 12:56:20 GMT

Good day. Consider me a noob who can google himself out of (and sometimes into) trouble.
Below I am logging my adventure on trying to upgrade my server, and running into "Passthrough not capable" error.

Feel free to post advice, or a solution if you see something wrong!

A few years ago I set up ESXi 6.7 on my desktop - it had a few linux VM's and one Windows VM with the GPU passed through to it. Everything was working great until I decided to upgrade.

Original Specs:

Mainboard: X570 Phantom Gaming 4 (BIOS versoin 1.20)
CPU: AMD 3600X (6 core)
MEM: 32GB 3200 mhz
GPU 1: Quadro K4000
GPU 2 : RTX 2060 OC

Upgraded Specs:
Mainboard: X570 Phantom Gaming 4 (BIOS version L5.01)
CPU: AMD 3700G (8 core)
MEM: 64GB 3200 mhz
GPU 1 : RTX 3080 TI

A couple days ago I upgraded the GPU to a 3080 TI. Restarting, I could see it was available for passthrough.
As the 3600X did not have built in graphics, I had an old Quadro K4000 for the graphics which the ESXi was using.
The Quadro was very close and blocking airflow to the 3080 ti, so I opted to upgrade the GPU to a 5700G, to allow for integrated graphics. When I switched GPU, I got an error code, google informed me that because I had bought my mainboard before the 5000 series CPU's were released, it needed a BIOS update. I put back the old CPU, did the BIOS update and then reinstalled the new 5700G. Everything worked, but BIOS on factory settings - memory at 2100 mhz or something low - and SR-IOV disabled.

Besides the output of display port being a pain (it wont send out a signal unless a monitor is detected, monitor will autoswitch to other inputs if no signal detected....) and just spamming F2 to get to BIOS before I could see anything, it would work. If i just left it to boot up, I could see the ESXi mainscreen (had to switch primary output to integrated graphics in BIOS) and I can connect to the server over my LAN as usual. However it now said SR-IOV and Passthrough "Not capable" and greyed out, where before it was black and said "disabled" and could be changed to "passthrough active".

I went back into the BIOS, I set the memory speed to 3200, and enabled SR-IOV. Reboot and I am still met with SR-IOV and Passthrough "Not capable" and greyed out.

I am thinking there may be some other BIOS settings I need to change, but this mainboard has a seriously complicated BIOS.
https://download.asrock.com/Manual/X570%20Phantom%20Gaming%204.pdf

Looking at it it looks like I need to ensure SVM is enabled. When I get get a chance I will verify this, but it says it should be enabled by default.

Other BIOS settings I am not clear if I should enable:
Above 4G Decoding "Enable or disable 64bit capable Devices to be decoded in Above 4G Address Space
(only if the system supports 64 bit PCI decoding)."

I can't see any other BIOS options. Could it be the L5.01 BIOS doesnt support Passthrough?

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

radixbl — Sat, 22 Apr 2023 12:01:18 GMT

Problem Resolved

had to go in BIOS to AMD CBS\NBIO and enable IOMMU.

Works flawlessly.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

cobaltblue1666 — Sat, 06 May 2023 20:27:09 GMT

This saved my bacon today! Thanks for sharing the details.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

radixbl — Tue, 09 May 2023 00:48:50 GMT

Awesome, glad it helped. I couldn't find anything online myself.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

cool_breeze — Wed, 10 May 2023 17:45:04 GMT

This was a helpful post.

topic Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update. in ESXi Discussions

ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

Re: ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.