I'm trying to automate the assignment of Nvidia Grid K1 GPUs via PCI passthrough for one of my vmware view pools. We are taking advantage of vdga for a 3D engineering class of about 30 seats and manually managing the cards via vcenter web console is a bit tedious. We are running vsphere and ESXI 5.5 and powercli version 5.8 release 1.
Looking at the example in the documentation for adding a passthrough device to a VM via powercli, I see that the command Add-PassthroughDevice is passed the an object that was had been returned from Get-PassthroughDevice. The problem that I'm running into is that the Get-Passthrough device will only return one object, the first GPU core, when there are four GPUs/Passthrough devices that can be assigned.
Would anyone have a suggestion as to how I would go about assigning the other three GPUs on bus 8, 9 and 10 via powercli? Is there something that I am overlooking?
(Get-EsxCli -VMHost "10.0.6.181").hardware.pci.list("0x0300") | select -Property Bus,CurrentOwner,DeviceName
Bus | CurrentOwner | DeviceName |
---|---|---|
7 | VM Passthru | NVIDIAGRID K1 |
8 | VM Passthru | NVIDIAGRID K1 |
9 | VM Passthru | NVIDIAGRID K1 |
10 | VM Passthru | NVIDIAGRID K1 |
16 | VMkernel | G200eR2 |
Get-PassthroughDevice -VMHost 10.0.6.181 -Type pci | where Name -EQ "NVIDIAGRID K1"
Bus | ClassId | DeviceId | Function | Slot | State | VendorId | VendorName | Name | VMId | VM | VMHostId | VMHost | Key | ExtensionData | Client |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
7 | 768 | 4082 | 0 | 0 | Active | 4318 | NVIDIA Corporation | NVIDIAGRID K1 | HostSystem-host-19 | 10.0.6.181 | VMware.Vim.VirtualMachinePciPassthroughInfo | VMware.VimAutomation.ViCore.Impl.V1.VimClient |
Did you figure out how to assign the other Grid GPU's? I am in the same situation right now.
I haven't found a solution using powershell. I don't think there is a way around the limitations of powercli. After posting this question, I did come across this post Pcipassthru which suggests using Orchestrator. I have yet to configure orchestrator and test this solution for myself. I'd be really interested to hear whether this works for you or not.
Good luck
Hi,
There no official solution. I used to have the same issue, and I fixed it using a makeshift PowerShell script. You may need to tweak it a bit for your needs. Please read the comments they are for you
[Console]::ResetColor()
. .\passtest.ps1
$GPUsIdslist= "0000:0a:00.0", "0000:09:00.0", "0000:08:00.0", "0000:07:00.0" # list the IDs of the PCI devices in the host starting from the bottom one. i am still working on making it
a non static entry for those IDs to make them auto-generated for each host. I using the command you provided in the original question to do so. Not yet ready sorry.
$destHostList1 = "10.0.6.181","10.0.6.182","10.0.6.183","10.0.6.184"
# Change depending on the ESXI hosts you have revers between list one and two this script hates using the same hosts for two successive VMs. Don't ask me why. I have no idea.
$destHostList2 = "10.0.6.184","10.0.6.183","10.0.6.182","10.0.6.181"
$operationloop = $vm_nbr # fill in the number of VMs
$Org_VMName = "VMtest" # fill in the common name between all VMs example: VMtest-01,VMtest-02,... (VMtest)
$j=0
$i=1
while ($i -le $operationloop)
{
$destVMName = $Org_VMName + "-" + $i.ToString("00") # + "-"
$j=$j+1
$VMName = $destVMName
write-host -ForeGroundColor Yellow "`n############ Operation Nbr: $j On: $VMName ##########`n"
# Adding PCi device GPU
Get-VM $VMName | where { $_.PowerState –eq "PoweredOn" } | Stop-VM –confirm:$false
Start-Sleep -s 5
foreach ($vm in (get-vm $VMName)) {get-vmresourceconfiguration $vm | set-vmresourceconfiguration -MemReservationMB $vm.MemoryMB}
$GPUID = "null"
get-vm $VMName | get-passthroughdevice | remove-passthroughdevice -Confirm:$false
$newvm = Get-vm $VMName
$destHost = $newvm.VMHost.Name
#get VMs info from the host
$GpuConf=get-vmhost $destHost | get-vm | get-view
$IdList = $GpuConf.config.hardware.device | ?{$_.Backing -is "VMware.Vim.VirtualPCIPassthroughDeviceBackingInfo"} | Select-Object -Property @{N="Id";E={$_.Backing.Id}}
$SlotLeft = 4 - $IdList.Id.Count
"-----------------------------------------------------"
Write-Host -ForeGroundColor Yellow "[DEBUG] The Host '$destHost' has '$SlotLeft' GPU slots left."
"-----------------------------------------------------"
foreach ($Idline in $GPUsIdslist)
{
if ($IdList -ne $null)
{
if (!($IdList.Id.Contains($Idline)))
{
$GPUID = $Idline
#continue
}
}
else
{
$GPUID = $GPUsIdslist[3]
}
}
if ($GPUID.Equals("null"))
{
"-----------------------------------------------------"
Write-Host -ForeGroundColor Red "`n[ERROR] The host '$destHost' has no free GPU. Please Migrate the VM '$VMName' to a host with available GPUs.`n"
"-----------------------------------------------------"
#$operationloop = $i
continue
}
"-----------------------------------------------------"
Write-Host -ForeGroundColor Green "[INFO] PCI Device with ID '$GPUID' from the host '$destHost' is going to be added to the VM '$destVMName'"
"-----------------------------------------------------"
add-uniquepcipassthroughdevice $VMName $GPUID $destHost
Start-Sleep -s 3
$g=get-view -viewtype VirtualMachine -filter @{"Name"=$VMName}
$h=$g.config.hardware.device | ?{$_.Backing -like "*Pass*"}
$h.backing
$gpuvm = Get-VM $newvm
$device = Get-PassthroughDevice -VM $gpuvm -Type Pci
$devname = $device.Name
$devid = $h.backing.Id
"-----------------------------------------------------"
Write-Host -ForeGroundColor Green "[INFO] PCI Device '$devname' with ID: ' $devid ' has been added to the VM '$newvm'"
Write-Host -ForeGroundColor Green "#################################################################################`n"
"-----------------------------------------------------"
foreach ($vm in (get-vm $newvm)) {get-vmresourceconfiguration $vm | set-vmresourceconfiguration -MemReservationMB $vm.MemoryMB}
Start-Sleep -s 2
# Start the VM
Start-VM $VMName
}
I hope this will help resolve your issue.
Mo
Can some one please help in power cli to add specific vgpu device to a VM.
I would like give input for VM name and vGPU profile and script should be able to add it.@