ESXI 6.7 and Nvidia GPUs

[ely105]

Hi all,

Just a quick post as a work in progress...

Now I know the first reply to this post will be something like, "Well it's unsupported hardware...".  And I want to thank you in advance for all the help.

Ok, enough snarking for today.

I've noticed GPU passthrough issues with ESXI 6.7 when I had things working with 6.5+, and it seems others are seeing this as well.

I tried a couple different hosts with different NVIDIA cards (1030, 1080ti) and got the same results, after seeing the card initialize it would shut down/crash.  From that point forward the dreaded error code 43 on device properties.  I read many posts in the past that said 43 was an intentional disable in software when the driver detected it was running in a VM.  So it got me thinking...

I always made sure that I had "hypervisor.cpuid.v0=FALSE" in my config before ever passing through the video card.  But something changed in 6.7, or maybe Nvidia is looking for something else?

So I tried this:

Build a Windows10 VM (v1809)

don't install vmware tools

install Chocolatey (https:\\chocolatey.org) For easier install of teamviewer and nvidia drivers

install teamviewer via chocolatey (so I can connect remotely) *also I didn't have usb hardware that I could passthrough

Disable svga(vmware vga) adapter "svga.present=FALSE" in vm advanced config

set "hypervisor.cpuid.v0=FALSE"

Add 2 PCI devices: 1080TI, Audio Device

boot while simultaneously crossing fingers.

Ok so the start was not pretty as Windows detects the video card and you need to reboot after initial install and you don't have vmware tools installed so its a cringe-worthy moment.

but alas after the reboot and the driver took, the output looked stable.

after that I installed the latest nvidia drivers via chocolatey (geforce-game-ready-driver) and all seems ok.

Quick testing with Cinebench and Unigine Superposition seems to work fine with a 1080p medium windowed score of 18390.

So, what was it?  svga present? vmware tools?

If I had to guess it might be svga, because I saw a similar issue with Ubuntu 18.10.  I was having problems getting the nvidia driver to work in Ubuntu so on a fluke I disabled the svga device and voila!  Ubuntu 18.10 on a GPU!

I may install vmware tools and see if that makes a difference.

Anyways, I wanted to throw this out there for all the folks suffering with this issue.  If it's Nvidias doing then I suppose it'll just be a matter of time for them to find another way.

-m

[IvarHome]

I have AMD video and dont need at all any special settings. ESXi is 6.5, drivers are card own drivers, windows 8.1, svga (ESXi own interface to VM) is disabled through windows settings (I dont need it and something remember it start some bad influence). Also soundcard and USB card through passthrough. But first I experimented with Nvidia card. I tried all settings suggested in internet, also this cpuid, no any help. The problem was with Nvidia drivers - they just refuse to install and thats all, passthrough worked but probably not 1:1, something missed and drivers refused to install. I think its because Nvidia wants your money, to buy GRID card........But the next step or hack sould be good if someone finds out how to make online snapshots or suspend. Its hard. All hardware components have hardware state. When you use VM or just use physical computer suspend/hibernate, then hardware state also saved. Thats because suspend works in laptops better than in workstations.

[ely105]

Yes it does seem to be an Nvidia driver *refusal* issue.

Radeon cards seem to passthru ok but it seems a bit more work to get them to play nice with ubuntu, although i was trying to get a Vega M GPU in Hades Canyon to work and never got the drivers to play either.

I would like to see if its possible to get a Vega 64 passthrough to work on a Mojave based VM.  Not sure if anyone has tried that, but...

-m

[Jellyman261]

Hello ely105, I just like you have been going from forum to forum trying to fix this issue when using a GTX 1070 from one of my old gaming rigs. I wondered if you remember the Nvidia driver version you use from your post above or maybe the one you currently use that is working today.

Thank you

JellyMan261