GPU passthru with 64-bit io map in ESXi problem

[AlekseyRyabkov]

Hello!

We are using ESXi 5.5 and try to passthru GPU to VM but have some trouble ....

1 . We have 2 GPU (AMD Firepro S10000 and nVidia Tesla K40) and SuperWorkstation 7047GR-TPRF (Supermicro | Products | SuperWorkstation | 4U | 7047GR-TPRF with X9DRG-QF motherboard)

2. nVidia Tesla K40 can work only when we set in BIOS support of 64-bit IO map (4G enable)

3. AMD Firepro S10000 can work with both 64-bit and 32-bit mode (4G enable/disable)

4. When we try to passthru AMD Firepro S10000 in  32-bit mode everything work fine

5. When we try to passthru AMD Firepro S10000  or nVidia Tesla K40 in  64-bit mode we have problem (similar to this https://communities.vmware.com/message/2316504)

But we cant use 32-bit mode (PC not boot with nVidia Tesla K40 in 32-bit mode - error is insufficient PCI resources and of course we can not boot with 2 cards)...

Any idea how to fix it?

By the way we try to use latest KVM and its works in this (64-bit) mode...

Thanks in advance, Aleksey Ryabkov

[antonionavas]

Hi, i've the same scenario (with Dell R720 and k40) and i can start the virtual machine, but the k40 appears in device manager with warning and doesn't start

[AlekseyRyabkov]

Hello!

1. We switch to Xen (its work for our AMD FirePro S10000 in 64-bit mode)... I dont know why AMD not work stable in 64-bit in ESXi...

2. Support form Nvidia told us that this is Supermicro BIOS bug (because its map MMAP BAR to address higher than 42-bit)... So we wrote letter to Supermicro ...

  - to see your assignment of BAR you can use lspci ....

With best regards, Aleksey Ryabkov

[iforbes]

Hi Alexsey. I'm having same issue with SuperMicro server and Tesla K40m card (MMAP BAR to address higher than 42-bit). Did you talk to SuperMicro and get a resolution?

Thanks

[iforbes]

Hi antonionavas. I'll be installing 2 x TeslaK40's into my DellR720 soon. Did you get a resolution to get things working?

Ian