When we run Get-VM | Get-TagAssignment | Select Entity,Tag in PowerCli we get this error:
HTTP response with status code 429 and no content (enable debug logging for details)
We have installed latest Powershell and PowerCli, but same problem.
429 means that vCenter gets to many requests, but we only have 1500vm´s, so can that really be true?
The above cmd, is quiet normal, and we use to be able to run it without errors.
Is there a place where i can see logs regarding PowerCLI calls to vCenter (logs on vCenter)?
Wow can i fix the problem?
Maybe an other way to get the same result?
You can check the log file "vpxd-svcs.log" to see the details.
Regards
Lokesh
Thanks for pointing me to the right log LokeshHK.
I see lots of errors in the log when i run Get-VM | Get-TagAssignment | Select Entity,Tag
2019-06-28T09:54:22.447+02:00 [tomcat-exec-209 ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter opId=] Illegal associableType : StoragePod
2019-06-28T09:54:22.447+02:00 [tomcat-exec-209 ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter opId=] Illegal associableType : com.vmware.content.library.Item
2019-06-28T09:54:22.447+02:00 [tomcat-exec-209 ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter opId=] Illegal associableType : com.vmware.content.Library
2019-06-28T09:54:22.447+02:00 [tomcat-exec-209 ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter opId=] Illegal associableType : VirtualMachine
I was told by support that when you delete a VM it doesn't remove the Tags and after a while you have a bunch of trash tags in the inventory which caused a loop and you get 429 error. support got in and stopped some vcenter services and ran a cleanup and it fixed my issue. Currently, they said a fix for the issue is not available at this time so it will happen again. Another solution is to remove the Tags before you remove the VM.
If you can shoot me your SR#, I would like to pass this along to our TAM. I wrote a fix that wraps and replaces Get-TagAssignment with a try/catch that loops until Get-TagAssignment returns without error, sleeping in between tries. This eliminated errant retrieval requests, but now I have to play with the sleep interval to avoid throttling.
<#
.SYNOPSIS
This function retrieves the tag assignments of objects
.DESCRIPTION
This function wraps the VMware.VimAutomation.Core cmdlet, Get-TagAssignment.
The VMware cmdlet has an intermittent and random error that causes it to not
return tag assignments. This wrapper detects those occurrences and retries
up to the number of times specified in Retries, waiting DelayMilliSeconds
before trying each time.
.NOTES
Author: Paul Knight
Works only with vCenter Server 5.1 or later.
.LINK
Online version: https://code.vmware.com/doc/preview?id=6330#/doc/Get-TagAssignment.html
Get-Tag
Get-TagAssignment
New-TagAssignment
Remove-TagAssignment
.PARAMETER DelayMilliSeconds
Number of milliseconds between retries if unable to retrieve the tag
assignment. Default is 250ms.
.PARAMETER Entity
Retrieves the tags associated with the specified items.
.PARAMETER Category
Returns the tags that belong to the specified categories.
.PARAMETER Retries
The number of retries to attempt before giving up. Failure is quiet and
returns null. Future revisions may want to make the nature of the failure
an option.
.PARAMETER Server
Specifies the vCenter Server systems on which you want to run the cmdlet. If
no value is passed to this parameter, the command runs on the default
servers. For more information about default servers, see the description of
Connect-VIServer.
.INPUTS
.OUTPUTS
Zero or more TagAssignment objects
.EXAMPLE
PS> $datastore = Get-DataStore MyDatastore
PS> Get-TagAssignment -Entity $datastore -Category MyCategory
Retrieves all tag assignments for the $datastore entity that have tags from
the "MyCategory" category.
#>
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline=$true)]
[PSObject]$Entity=@($null),
[PSObject[]]$Category,
[PSObject[]]$Server,
[int]$DelayMilliSeconds=250,
[int]$Retries=500
)
begin {
"Started execution" | Out-Verbose -Caller Get-TagAssignmentDTI
}
process {
foreach ($obj in $Entity) {
$tries = 0
$bRetry = $true
while ($bRetry -and $tries -le $Retries) {
$tries++
try {
VMware.VimAutomation.Core\Get-TagAssignment -Entity $obj -Category $Category -Server $Server -ErrorAction Stop -Verbose:$false
$bRetry = $false
} catch [VMware.VimAutomation.Cis.Core.Types.V1.CisException] {
# Resolve-Error($_)
if ($obj) {
("Retrying {0} {1}" -f $obj.Name,$tries) |
Out-Verbose -Caller Get-TagAssignmentDTI -Verbosity "High"
} else {
("Retrying {0}" -f $tries) |
Out-Verbose -Caller Get-TagAssignmentDTI -Verbosity "High"
}
Start-Sleep -m $DelayMilliSeconds
}
}
}
}
end {
"Finished execution" | Out-Verbose -Caller Get-TagAssignmentDTI
}
}