Trouble vDGA: Nvidia and ESXi 5.5

[mgordeev]

Hello! Good day!

I have Nvidia Tesla k40 and SuperMicro x9drg-qf.  My server is having 2 CPU and 64GB RAM, PCIe 3.0.

I will try power up my Virtual Machines in passthroug mode. But I receive error:

PCIPassthru: 02:00.0 : Device BAR 0 requested 64-bit memory address that exceeded MPN type

PCIPassthru: 02:00.0 : Device BAR 4 requested 64-bit memory address that exceeded MPN type

BIOS SuperMicro is having option: Memory Mapped I/O above 4GB == Enabled.  I can`t change this option to Disable, Tesla will not work (no POST message in BIOS, Insufficient PCI Resources).

How I can use vDGA mode in my VM in this configurations.

I thank you for answers.

[Linjo]

Check that you have the latest available bios from SuperMicro and the latest patches for vSphere.

If there is still a problem start with posting the vmx-file from the vm and the vmware.log.

// Linjo

[mgordeev]

Hi, Linjo!

Yes, my system is update!

~ # uname -a

VMkernel cheap2.ntc-vulkan.ru 5.5.0 #1 SMP Release build-1623387 Feb 21 2014 17:19:17 x86_64 GNU/Linux

Please, look at the attched files!

Thanks!

[mgordeev]

UP! 🙂

[mgordeev]

UP!

[NTStech]

Surprised nobody has answered this for you yet:

VMware KB: Systems with MMIO regions larger than 4GB are not supported

[iforbes]

Hi mgordeev. How did you get this to work? I'm in the same boat with Supermicro and Tesla K40 card.

Thanks

[mgordeev · Accepted solution]

Hello, Iforbes! I am using qemu-kvm. Kvm is working with k40m! I have two cards(k40c and k40m). Please, look at this specification: http://international.download.nvidia.com/tesla/pdf/tesla-k40-passive-board-spec.pdf (6 page) http://www.nvidia.ru/content/PDF/kepler/Tesla-K40-Active-Board-Spec-BD-06949-001_v03.pdf (7 page) This information was listed in nvidia specification and I can see in output by command "lspci": Tesla k40c - BAR1 256M Tesla k40m - BAR1 16gb. BAR1 is very big for Xen and VMware... It can not maping the memory... May be Xen fixed this problem... I don't know. I wish you an easy fix of your problem! You can write me on my email.

[iforbes]

Hi mgordeev. Yah it seems that vSphere cannot handle MMIO above 4GB (VMware kb 2050443). I'm working with SuperMicro to see if there is a workaround.