VMware Cloud Community
vmrulz
Hot Shot
Hot Shot

NVIDIA GRID vGPU - Rocky start, any good experiences/help?

Greetings,

We are attempting to configure dual Tesla M10 GPU cards in our VxRail E570F servers (Dell 14G 2U box) to be used for a Citrix Xendesktop VDI cluster. We are finding that we cannot get ESXi to properly allocate memory to these cards with the default MMIO setting = 56T. Changing the setting to 12T or 512G per this Dell article causes the server to purple screen a few minutes after full boot.

This is using the latest available VIB from NVIDIA on ESXi 6.5u3c.

https://www.dell.com/support/article/ht/en/htdhs1/sln308065/dell-poweredge-14g-esxi-returns-failed-t...

NVIDIA support portal is down so I'm attempting to get knowledge elsewhere on this tech.

Thanks for any advice.

Reply
0 Kudos
5 Replies
vmrulz
Hot Shot
Hot Shot

Interesting lack of responses on this tech. We found that a card was not properly seated causing the PSOD. We still however have a problem with the card and it is being replaced.

Should be an interesting adventure.

Reply
0 Kudos
flynmooney
Enthusiast
Enthusiast

We had a NVidia Tesla M6 card in a host running 6.0U3 for a while.  It was a total pain to setup and get working.   I had to enlist both VMware and NVidia for support to finally get it working.  If I remember right there was something messed up in a the xorg config file which VMware support found and sent me an updated file.

Reply
0 Kudos
vmrulz
Hot Shot
Hot Shot

Yeah and the caveats continue to pop up.. The current incarnation of Nvidia drivers and ESXi do not allow for VMware HA or DRS or vmotion. So throw out all the things we've become accustomed for HA and load balancing for better graphics performance. Argh.

Reply
0 Kudos
Dave_the_Wave
Hot Shot
Hot Shot

So basically who is really doing this?

Does this mean Nvidia has sold about total 10 of these cards worldwide?

Maybe that's why they are so expensive. You order one, pay a deposit, and they go build one for you like a Maybach.

I'm getting the feeling the buyer pays tens of thousands of dollars to join a hardware beta program.

Reply
0 Kudos
TheBobkin
Champion
Champion

Hello vmrulz​,

"The current incarnation of Nvidia drivers and ESXi do not allow for VMware HA or DRS or vmotion.

Yes, as you are essentially using passthrough of a (part of a) piece of hardware - making a functional method of 'vMotioning' a current state graphics card IO (without interruption) isn't a trivial task.

"So throw out all the things we've become accustomed for HA and load balancing for better graphics performance. Argh."

Sorry to hear that you are frustrated regarding this, but I assume you have been in IT some time and understand that in many spheres often benefits/features come with trade-offs/caveats based on the fundamentals of how these features function (e.g. expecting 10x compression with no negative IOPS impact).

@Dave_the_Wave

"So basically who is really doing this? Does this mean Nvidia has sold about total 10 of these cards worldwide?"

Actually they are fairly common - look at any Horizon View cluster or similar VDI product (schools/colleges is a big market) and they will generally be utilising GRID cards.

Bob

Reply
0 Kudos