NIC passthrough causes network instability

jackgagliano136 · ‎08-07-2022

Hi All,

I have a supermicro server with 4 NIC's on board. I have 1 NIC for the management network, and 2 NIC's I want to passthrough to a VM for direct access. I go into hardware and enable the passthrough for the 2 that I want and they are listed as Active (see attached).

When I power on the VM, it powers on just fine and everything inside the VM operates as normal, except for when the NICs become active, it totally destabilizes the management network to esxi. I have verified this by running a ping against the management NIC continuously and can see the moment the VM's NICs become active, I no longer have "stable" access to the management network. I have also tried putting the passed through NICs in a Windows 10 VM and the same thing happens there, so I know it's not the VM that was doing it. My question.. is it possible that vmware can't separate these NICs? I'm wondering if it is like an all or nothing.. everything in the IOMMU group has to get passed through or nothing, even though it is allowing me to pass through individual NICs. Also, when viewing the ping status, if I shutdown the VM.. the ping resumes normally.

edit: Just re-read what I wrote.. No, I am not passing through the NIC that is on the management network, just in case anyone was wondering. There are spots of activity where I can do things, but it is really not stable when the other NICs are passed through.

Have been beating my head on this all day.. hopefully someone with some fresh eyes can give me some insight.

Thanks!

Jack

bluefirestorm · ‎08-07-2022

The 4 NICs are on the same PCIe bus (0b) so there could be issues when a reset command is issued. As the NICs shares the same bus, it is also better to passthrough these NICs to the same VM rather than to different VMs.

jackgagliano136 · ‎08-08-2022

Hey! Thanks for replying! I was afraid of that.. I looked up the i350 and it is supposed to support FLR. I even went and grabbed the latest driver that vmware had and installed it thinking it would help, it didn't. I did some other testing though.. it seems that I can passthrough one of the NICs with no issue.. if I pass through the 0 NIC it has the same instability issues.. however if I passthrough the 1 NIC it is fine, with both it has the same instability issues. So I am guessing that is exactly what you are saying is that for some reason vmware can't separate them even though it says it can.