VMware Cloud Community
vmwarelicense12
Enthusiast
Enthusiast

Get-TagAssignment fails with status code 429

When we run Get-VM | Get-TagAssignment | Select Entity,Tag in PowerCli we get this error:

HTTP response with status code 429 and no content (enable debug logging for details)

We have installed latest Powershell and PowerCli, but same problem.

429 means that vCenter gets to many requests, but we only have 1500vm´s, so can that really be true?

The above cmd, is quiet normal, and we use to be able to run it without errors.

Is there a place where i can see logs regarding PowerCLI calls to vCenter (logs on vCenter)?

Wow can i fix the problem?

Maybe an other way to get the same result?

0 Kudos
5 Replies
LokeshHK
VMware Employee
VMware Employee

You can check the log file "vpxd-svcs.log" to see the details.

Regards

Lokesh

0 Kudos
vmwarelicense12
Enthusiast
Enthusiast

Thanks for pointing me to the right log LokeshHK.

I see lots of errors in the log when i run Get-VM | Get-TagAssignment | Select Entity,Tag

2019-06-28T09:54:22.447+02:00 [tomcat-exec-209  ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter  opId=] Illegal associableType : StoragePod

2019-06-28T09:54:22.447+02:00 [tomcat-exec-209  ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter  opId=] Illegal associableType : com.vmware.content.library.Item

2019-06-28T09:54:22.447+02:00 [tomcat-exec-209  ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter  opId=] Illegal associableType : com.vmware.content.Library

2019-06-28T09:54:22.447+02:00 [tomcat-exec-209  ERROR com.vmware.cis.core.tagging.internal.helper.TaggingConverter  opId=] Illegal associableType : VirtualMachine

0 Kudos
progsaz
Contributor
Contributor

I was told by support that when you delete a VM it doesn't remove the Tags and after a while you have a bunch of trash tags in the inventory which caused a loop and you get 429 error.  support got in and stopped some vcenter services and ran a cleanup and it fixed my issue.  Currently, they said a fix for the issue is not available at this time so it will happen again.  Another solution is to remove the Tags before you remove the VM.

0 Kudos
Paul_Knight1
Contributor
Contributor

If you can shoot me your SR#, I would like to pass this along to our TAM.  I wrote a fix that wraps and replaces Get-TagAssignment with a try/catch that loops until Get-TagAssignment returns without error, sleeping in between tries.  This eliminated errant retrieval requests, but now I have to play with the sleep interval to avoid throttling.

0 Kudos
Paul_Knight1
Contributor
Contributor

function Get-TagAssignmentDTI {

<#

.SYNOPSIS

  This function retrieves the tag assignments of objects

.DESCRIPTION

  This function wraps the VMware.VimAutomation.Core cmdlet, Get-TagAssignment.

  The VMware cmdlet has an intermittent and random error that causes it to not

  return tag assignments. This wrapper detects those occurrences and retries

  up to the number of times specified in Retries, waiting DelayMilliSeconds

  before trying each time.

.NOTES

  Author: Paul Knight


  Works only with vCenter Server 5.1 or later.

.LINK

  Online version: https://code.vmware.com/doc/preview?id=6330#/doc/Get-TagAssignment.html

  Get-Tag

  Get-TagAssignment

  New-TagAssignment

  Remove-TagAssignment

.PARAMETER DelayMilliSeconds

  Number of milliseconds between retries if unable to retrieve the tag

  assignment. Default is 250ms.

.PARAMETER Entity

  Retrieves the tags associated with the specified items.

.PARAMETER Category

  Returns the tags that belong to the specified categories.

.PARAMETER Retries

  The number of retries to attempt before giving up. Failure is quiet and

  returns null. Future revisions may want to make the nature of the failure

  an option.

.PARAMETER Server

  Specifies the vCenter Server systems on which you want to run the cmdlet. If

  no value is passed to this parameter, the command runs on the default

  servers. For more information about default servers, see the description of

  Connect-VIServer.

.INPUTS

.OUTPUTS

  Zero or more TagAssignment objects

.EXAMPLE

  PS> $datastore = Get-DataStore MyDatastore

  PS> Get-TagAssignment -Entity $datastore -Category MyCategory


  Retrieves all tag assignments for the $datastore entity that have tags from

  the "MyCategory" category.

#>

[CmdletBinding()]

param (

  [Parameter(ValueFromPipeline=$true)]

  [PSObject]$Entity=@($null),

  [PSObject[]]$Category,

  [PSObject[]]$Server,

  [int]$DelayMilliSeconds=250,

  [int]$Retries=500

)


  begin {

    "Started execution" | Out-Verbose -Caller Get-TagAssignmentDTI

  }


  process {

    foreach ($obj in $Entity) {

    $tries = 0

    $bRetry = $true

    while ($bRetry -and $tries -le $Retries) {

      $tries++

      try {

        VMware.VimAutomation.Core\Get-TagAssignment -Entity $obj -Category $Category -Server $Server -ErrorAction Stop -Verbose:$false

        $bRetry = $false

      } catch [VMware.VimAutomation.Cis.Core.Types.V1.CisException] {

#        Resolve-Error($_)

        if ($obj) {

          ("Retrying {0} {1}" -f $obj.Name,$tries) |

            Out-Verbose -Caller Get-TagAssignmentDTI -Verbosity "High"

        } else {

          ("Retrying {0}" -f $tries) |

            Out-Verbose -Caller Get-TagAssignmentDTI -Verbosity "High"

        }

        Start-Sleep -m $DelayMilliSeconds

       }

      }

    }

  }


  end {

    "Finished execution" | Out-Verbose -Caller Get-TagAssignmentDTI

  }

}

0 Kudos