JLogan2016
Enthusiast
Enthusiast

Working code suddenly stopped working

I have a server build script that has been running fine for months. A week or so ago I began to notice errors in one section of the code, specifically a function that applies tags after the server is built. The original function looks like so:

function _addTagging{

    [CmdLetBinding()]

    Param (

        [Parameter(Mandatory, Position=1)]

            [string] $newVM,

        [Parameter(Mandatory, Position=2)]

            [string] $OS,

        [Parameter(Mandatory, Position=3)]

            [string] $Env,

        [Parameter(Mandatory, Position=4)]

            [string] $LOB,

        [Parameter(Mandatory, Position=5)]

            [string] $Owner,

        [Parameter(Mandatory, Position=6)]

            [string] $iGroup

    )

    $aCategories = @('Operating System Architecture', 'Environment', 'Line of Business', 'Owner', 'OnCallGroup')

    $aTags = @($OS, $Env, $LOB, $Owner, $iGroup)

    if ($newVM[2] -eq "O") {

        $vCenter = "Location1VCSA"

    } else {

        $vCenter = "Location2VCSA"

    }

    foreach ($a in 0..4) {

        try {

            $sVM = Get-VM -Name $newVM -ErrorAction Stop | Select Name, @{N='vCenter';E={([System.Uri]$_.ExtensionData.Client.ServiceUrl).Host}}

            foreach ($a in 0..4) {

                $tag = Get-Tag -Name $aTags[$a] -Category $aCategories[$a] -Server $sVM.vCenter

                try {

                    New-TagAssignment -Tag $tag -Entity $sVM.Name -Server $sVM.vCenter -ErrorAction Stop | Out-Null

                    _writeLog -Path $LogFile -Message "     $($aCategories[$a]) tag set to $($tag.Name)"  -Component "VM Build" -Type Info

                } catch {

                    _writeLog -Path $LogFile -Message "     Failed to add $($aCategories[$a]) Tag. Returned error: $($error[0])"  -Component "VM Build" -Type Warning

                    $iWarnings += 1

                }

            }

        } catch {

            _writeLog -Path $LogFile -Message $error.ToString()   -Component "VM Build" -Type Warning

            _writeLog -Path $LogFile -Message "     Could not find $($newVM), no Tagging will be done"  -Component "VM Build" -Type Warning

            $iWarnings += 1

        }

    }

}

The error I began receiving indicates the $tag is coming back null:

Get-Tag : 3/19/2020 12:04:02 PM Get-Tag Value cannot be null.

Parameter name: collection

At line:33 char:12

+     $tag = Get-Tag -Name $aTags[$a] -Category $aCategories[$a] -Serv...

+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Get-Tag], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Com

   mands.Tagging.GetTag

I have even tried to dumb the code down with a simple:

$tag = Get-Tag -Name "Prod" -Category "Environment" -Server $sVM.vCenter

I can see this tag, in this category, in the UI. However it still comes back with the error above. Trying to figure out why this suddenly stopped working after several months; hoping I am missing something stupid simple.

0 Kudos
7 Replies
LucD
Leadership
Leadership

Was you vSphere version upgraded perhaps?

Which vSphere version, which PowerCLI version are you using?
Can you eventually downgrade your PowerCLI version to 10.*, provided you are currently using 11.*?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
JLogan2016
Enthusiast
Enthusiast

No vCenter upgrades, but I am on 11.3.0 b13990089 on my desktop and 11.5.0 b14912921 on my automation server. Is this a known issue with 11? I can try spinning up a server and installing 10 on it to test.

0 Kudos
LucD
Leadership
Leadership

There have been some issues with earlier 11.* releases and Tag related cmdlets.
Perhaps first try upgrading to 11.5, but it looks as if your server already has that.

If that doesn't work, a downgrade to a 10.* version could be the next test.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
JLogan2016
Enthusiast
Enthusiast

Strangely enough, I went to another server, installed PowerCLI 11.5 and tried it, and everything worked just fine. Wonder if is comes down to a profile issue.

0 Kudos
LucD
Leadership
Leadership

Then it could also be a port 7444 issue (that is the port for the STS Token Service on the VCSA).

Check if you reach that port from the station where it fails


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
JLogan2016
Enthusiast
Enthusiast

I did some further testing last night; found on my desktop that if I ran PS with -noprofile it was working. So overnight I rebooted both my desktop and the automation server, and restarted vsphere-client, vsphere-ui, and vapi-endpoint services on the vCenter server. In testing this morning, on my desktop, all is working as it should (although I noticed authentication to vCenter is slow) with the code below. This works whether I authenticate to vCenter as myself or as the service account:

cls

$aHosts = @(

    "myvCenter"

)

Connect-VIServer -Server $aHosts -Credential (Get-Credential)

Get-Tag -Name "Windows Server 2016 Datacenter" -Category "Operating System Architecture"

Disconnect-VIServer -Server * -Confirm: $false

Remove-Variable * -ErrorAction SilentlyContinue; $error.Clear()

Output is what I would expect:

Category    : Operating System Architecture

Description :

Id          : urn:vmomi:InventoryServiceTag:7550f4e6-c349-4a07-acb1-6875740a687d:GLOBAL

Name        : Windows Server 2016 Datacenter

Uid         : /VIServer=mydomain\me@myvCenter:443/Tag=urn:vmomi:InventoryServiceTag:7550f4e6-c349-4a07-acb1-6875740a687d:GLOBAL/

On the server, logged in as myself, I get the same result, just what I would expect, whether supplying my credentials or that of the service account. Logged in as the service account, however, I still get the $tag is null error. This happens whether I run PS normally or with the -noprofile switch. So it really seems to be a profile issue; just bizarre that this is the only piece not working - the VM is built, the drives are expanded and formatted, etc. etc. I am going to blow away the Windows profile on the automation server this evening, and see if that resolves the issue.

0 Kudos
LucD
Leadership
Leadership

Strange indeed.

Do you load anything special in that profile?
The Tag cdmlets, in the latest PowerCLI versions, use the REST API.

Perhaps there is an issue with REST calls?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos