I have a server build script that has been running fine for months. A week or so ago I began to notice errors in one section of the code, specifically a function that applies tags after the server is built. The original function looks like so:
function _addTagging{
[CmdLetBinding()]
Param (
[Parameter(Mandatory, Position=1)]
[string] $newVM,
[Parameter(Mandatory, Position=2)]
[string] $OS,
[Parameter(Mandatory, Position=3)]
[string] $Env,
[Parameter(Mandatory, Position=4)]
[string] $LOB,
[Parameter(Mandatory, Position=5)]
[string] $Owner,
[Parameter(Mandatory, Position=6)]
[string] $iGroup
)
$aCategories = @('Operating System Architecture', 'Environment', 'Line of Business', 'Owner', 'OnCallGroup')
$aTags = @($OS, $Env, $LOB, $Owner, $iGroup)
if ($newVM[2] -eq "O") {
$vCenter = "Location1VCSA"
} else {
$vCenter = "Location2VCSA"
}
foreach ($a in 0..4) {
try {
$sVM = Get-VM -Name $newVM -ErrorAction Stop | Select Name, @{N='vCenter';E={([System.Uri]$_.ExtensionData.Client.ServiceUrl).Host}}
foreach ($a in 0..4) {
$tag = Get-Tag -Name $aTags[$a] -Category $aCategories[$a] -Server $sVM.vCenter
try {
New-TagAssignment -Tag $tag -Entity $sVM.Name -Server $sVM.vCenter -ErrorAction Stop | Out-Null
_writeLog -Path $LogFile -Message " $($aCategories[$a]) tag set to $($tag.Name)" -Component "VM Build" -Type Info
} catch {
_writeLog -Path $LogFile -Message " Failed to add $($aCategories[$a]) Tag. Returned error: $($error[0])" -Component "VM Build" -Type Warning
$iWarnings += 1
}
}
} catch {
_writeLog -Path $LogFile -Message $error.ToString() -Component "VM Build" -Type Warning
_writeLog -Path $LogFile -Message " Could not find $($newVM), no Tagging will be done" -Component "VM Build" -Type Warning
$iWarnings += 1
}
}
}
The error I began receiving indicates the $tag is coming back null:
Get-Tag : 3/19/2020 12:04:02 PM Get-Tag Value cannot be null.
Parameter name: collection
At line:33 char:12
+ $tag = Get-Tag -Name $aTags[$a] -Category $aCategories[$a] -Serv...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Get-Tag], VimException
+ FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Com
mands.Tagging.GetTag
I have even tried to dumb the code down with a simple:
$tag = Get-Tag -Name "Prod" -Category "Environment" -Server $sVM.vCenter
I can see this tag, in this category, in the UI. However it still comes back with the error above. Trying to figure out why this suddenly stopped working after several months; hoping I am missing something stupid simple.
Was you vSphere version upgraded perhaps?
Which vSphere version, which PowerCLI version are you using?
Can you eventually downgrade your PowerCLI version to 10.*, provided you are currently using 11.*?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
No vCenter upgrades, but I am on 11.3.0 b13990089 on my desktop and 11.5.0 b14912921 on my automation server. Is this a known issue with 11? I can try spinning up a server and installing 10 on it to test.
There have been some issues with earlier 11.* releases and Tag related cmdlets.
Perhaps first try upgrading to 11.5, but it looks as if your server already has that.
If that doesn't work, a downgrade to a 10.* version could be the next test.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Strangely enough, I went to another server, installed PowerCLI 11.5 and tried it, and everything worked just fine. Wonder if is comes down to a profile issue.
Then it could also be a port 7444 issue (that is the port for the STS Token Service on the VCSA).
Check if you reach that port from the station where it fails
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
I did some further testing last night; found on my desktop that if I ran PS with -noprofile it was working. So overnight I rebooted both my desktop and the automation server, and restarted vsphere-client, vsphere-ui, and vapi-endpoint services on the vCenter server. In testing this morning, on my desktop, all is working as it should (although I noticed authentication to vCenter is slow) with the code below. This works whether I authenticate to vCenter as myself or as the service account:
cls
$aHosts = @(
"myvCenter"
)
Connect-VIServer -Server $aHosts -Credential (Get-Credential)
Get-Tag -Name "Windows Server 2016 Datacenter" -Category "Operating System Architecture"
Disconnect-VIServer -Server * -Confirm: $false
Remove-Variable * -ErrorAction SilentlyContinue; $error.Clear()
Output is what I would expect:
Category : Operating System Architecture
Description :
Id : urn:vmomi:InventoryServiceTag:7550f4e6-c349-4a07-acb1-6875740a687d:GLOBAL
Name : Windows Server 2016 Datacenter
Uid : /VIServer=mydomain\me@myvCenter:443/Tag=urn:vmomi:InventoryServiceTag:7550f4e6-c349-4a07-acb1-6875740a687d:GLOBAL/
On the server, logged in as myself, I get the same result, just what I would expect, whether supplying my credentials or that of the service account. Logged in as the service account, however, I still get the $tag is null error. This happens whether I run PS normally or with the -noprofile switch. So it really seems to be a profile issue; just bizarre that this is the only piece not working - the VM is built, the drives are expanded and formatted, etc. etc. I am going to blow away the Windows profile on the automation server this evening, and see if that resolves the issue.
Strange indeed.
Do you load anything special in that profile?
The Tag cdmlets, in the latest PowerCLI versions, use the REST API.
Perhaps there is an issue with REST calls?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference