VMware Cloud Community
GMSS_LLP
Enthusiast
Enthusiast

Nvidia Tesla T4 GPU and ESXi 6.5

Hi Guys,

I've recently installed a Nvidia Tesla T4 GPU in my server running ESXi 6.5.

I've tried making a passthrough on all 32 addresses for the card on the list but when that happens and when and when I try to assign the card to a VM, the option the assign the PCI device is greyed out.

I'e managed to to set a passthrough and make Active 1 address and than I was managed to assign this card to the VM.

pastedImage_0.png

My question is:

1. Does this mean that only 1 GPU core is assigned to the VM out of a possible 32?

2. If so, any ideas how I'm able to assign all 32 cores of this card to the VM as when I try the option to add the PCI device to the VM is greyed out?

Please advise.

Kind Regards

GMSS

Reply
0 Kudos
6 Replies
berndweyand
Expert
Expert

and when you select one card and select "toggle passthrough" ?

t4.png

Reply
0 Kudos
GMSS_LLP
Enthusiast
Enthusiast

Hi Bewe,

Thanks for your reply.

As in the image when I select 1, passthrough turns Active and I'm able to attach the PCI device (which not greyed out) to the VM.

pastedImage_0.png

Does the different addresses of the Tesla T4 represent the cores? There are 32 different addresses does that mean that there are 32 cores on the Tesla T4?

If this is the case, will this mean that I've only activated 1 core and have only assigned 1 core the my VM?

Kind Regards

GMSS

Reply
0 Kudos
berndweyand
Expert
Expert

Hello,

i cannot test this because i have installed the nvidia-grid vib on the host and they are active.

just test if you can activate all cores and after reboot if you are able to assign them

Reply
0 Kudos
GMSS_LLP
Enthusiast
Enthusiast

Hi Bewe,

When I attempt to activate all cores for passthrough and after the reboot the Tesla T4 card cannot be seen and so adding the PCI device option is greyed out.

Upon rebooting the server again, the Tesla T4 card reappears and the passthrough is disabled on all cores. It's only when I select 1 core i'm able to make the

passthrough active and assign the PCI Device (not greyed out now) to the VM.

Any more ideas?

Kind regards

Keith

Reply
0 Kudos
berndweyand
Expert
Expert

here is an example that the card was in the wrong pic-slot https://serverfault.com/questions/950161/esxi-tesla-passthrough-enabled-but-not-assignable?rq=1

https://serverfault.com/questions/950161/esxi-tesla-passthrough-enabled-but-not-assignable?rq=1

i am not sure if the card is working without nvidia vib drivers because we have installed them and add a shared pci device to the vm - but therefor you need vcenter and not only a single esxi. i can not test it for you because all hosts with vgpu are productive

maybe its better to ask in the nvidia-forum ? https://gridforums.nvidia.com/

Reply
0 Kudos
GMSS_LLP
Enthusiast
Enthusiast

Hi Bewe,

Many thanks for your reply and links.

I have installed the T4 vib drivers on the ESXi host and only then I was able to passthrough and make active 1 address (out of the possible 32).

When I try to passthrough and make active all 32 and try to assign it to the VM, the PCI device option is still greyed out.  

pastedImage_0.png

This can't be normal can it? I should be able to passthrough all 32 and make active and assign it to the VM?

I've asked the question in the nvidia forum as advised (but no takers as yet) and appreciate all the help and pointers you've give thus far.

Kind Regards

GMSS

Reply
0 Kudos