VMware Cloud Community
radixbl
Contributor
Contributor

ESXI SR-IOV & Passthrough not capable for all devices after CPU switch and BIOS update.

Good day. Consider me a noob who can google himself out of (and sometimes into) trouble.
Below I am logging my adventure on trying to upgrade my server, and running into "Passthrough not capable" error.

Feel free to post advice, or a solution if you see something wrong!

A few years ago I set up ESXi 6.7 on my desktop - it had a few linux VM's and one Windows VM with the GPU passed through to it. Everything was working great until I decided to upgrade.

Original Specs:

Mainboard: X570 Phantom Gaming 4 (BIOS versoin 1.20)
CPU: AMD 3600X (6 core)
MEM: 32GB 3200 mhz
GPU 1: Quadro K4000
GPU 2 : RTX 2060 OC

Upgraded Specs:
Mainboard: X570 Phantom Gaming 4 (BIOS version L5.01)
CPU: AMD 3700G (8 core)
MEM: 64GB 3200 mhz
GPU 1 : RTX 3080 TI

A couple days ago I upgraded the GPU to a 3080 TI. Restarting, I could see it was available for passthrough.
As the 3600X did not have built in graphics, I had an old Quadro K4000 for the graphics which the ESXi was using.
The Quadro was very close and blocking airflow to the 3080 ti, so I opted to upgrade the GPU to a 5700G, to allow for integrated graphics.  When I switched GPU, I got an error code, google informed me that because I had bought my mainboard before the 5000 series CPU's were released, it needed a BIOS update. I put back the old CPU, did the BIOS update and then reinstalled the new 5700G. Everything worked, but BIOS on factory settings - memory at 2100 mhz or something low - and SR-IOV disabled.

Besides the output of display port being a pain (it wont send out a signal unless a monitor is detected, monitor will autoswitch to other inputs if no signal detected....) and just spamming F2 to get to BIOS before I could see anything, it would work. If i just left it to boot up, I could see the ESXi mainscreen (had to switch primary output to integrated graphics in BIOS) and I can connect to the server over my LAN as usual. However it now said SR-IOV and Passthrough "Not capable" and greyed out, where before it was black and said "disabled" and could be changed to "passthrough active".

I went back into the BIOS, I set the memory speed to 3200, and enabled SR-IOV. Reboot and I am still met with SR-IOV and Passthrough "Not capable" and greyed out.

I am thinking there may be some other BIOS settings I need to change, but this mainboard has a seriously complicated BIOS.
https://download.asrock.com/Manual/X570%20Phantom%20Gaming%204.pdf

Looking at it it looks like I need to ensure SVM is enabled. When I get get a chance I will verify this, but it says it should be enabled by default.

Other BIOS settings I am not clear if I should enable:
Above 4G Decoding "Enable or disable 64bit capable Devices to be decoded in Above 4G Address Space
(only if the system supports 64 bit PCI decoding)."

 

I can't see any other BIOS options. Could it be the L5.01 BIOS doesnt support Passthrough?

 

 

 

4 Replies
radixbl
Contributor
Contributor

Problem Resolved

had to go in BIOS to AMD CBS\NBIO and enable IOMMU.

Works flawlessly.

cobaltblue1666
Contributor
Contributor

This saved my bacon today!  Thanks for sharing the details.

0 Kudos
radixbl
Contributor
Contributor

Awesome, glad it helped. I couldn't find anything online myself.

07

0 Kudos
cool_breeze
Enthusiast
Enthusiast

This was a helpful post.

0 Kudos