VMware Cloud Community
GordonPM
Enthusiast
Enthusiast

Get-TagAssignment operation causes Disconnect-Viserver to hang

Ourbackup software uses the tags we assing in vSphere to schedule backups/type. One drawback of this is that sometimes adding a VM to backups is forgotten

I have written a small script using PS/PowrerCLI and windows Forms to list all the VMs, along with their backup tag.

The script works perfectly, to connect, amd I can list VMs, however if I use Get-TagAssignment to query the tag detail any following operation to disconnect from vCenter causes the entire Windows Form to hang.

Happy to provide the whole script if anyone wants to try and reporduce, as there's no sensitive info in it.

As an example there are two values for the variable $BackupTag in the following snippet - the one that lists the VM name works fine and the code/Windows Form closes gracefully, but if I run the second, querying the Tags, the Form will hang when I try to disconnect from vCenter:

$ScanForVMs.Add_Click({

    $VcConnectText.AppendText("`r`nRetrieving VM list")

    $Script:AllVMs = Get-VM | Sort-Object Name

    $TotalVMs.Text = [int]$AllVMs.count

    $VcConnectText.AppendText("`r`nDone")

    $VcConnectText.AppendText("`r`nRetrieving VM tags")

# Check if PoweredOff checkbox enabled

    If ($Script:ScanPoweredOff) {

        $Script:ScanVMList = $AllVMs

    }

    Else {

        $Script:ScanVMList = $AllVMs | Where-Object {$_.PowerState -ne 'PoweredOff'}

    }

# Compile object data

     $NoTagText = "No Backup Tag"

     $ScanVMList | ForEach-Object {

#     $BackupTag = $_.Name

     $BackupTag = ((Get-TagAssignment -Entity $_ ).Tag.Name).ToString() # | Where-Object {$_ -in $ValidTags.Name}

#

     $VcConnectText.AppendText("`r`n$BackupTag")
    }

    $VcConnectText.AppendText("`r`nDone")

})

# Detect if form is closing, if the form exit button hasn't been used, cancel the close and prompt so that we can disconnect cleanly from vCenter

$form.Add_FormClosing({

    If ($script:CanClose -ne $true) {

    $VcConnectText.AppendText("`r`nPlease use Exit button to disconnect from vCenter and close the application")

    $_.Cancel = $true

    }

#

# Exit and close form

$ExitButton.Add_Click({

    Try {

        Disconnect-VIServer -Server * -Force -Confirm:$false -ErrorAction Stop

    }

    Catch {


    }

    $script:CanClose = $true

    $form.close()

})

Reply
0 Kudos
15 Replies
LucD
Leadership
Leadership

That is difficult to analyse without the complete script (especially the form code).
Can you attach the complete code (no need to post it inline)


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

It defintiely seems related to tag retrieval - I've commented in the code where I added a get-tag command to retrieve the tag category for backups and use it to filter, but this alone causes the hang on disconnect. Just connect to VIServer and try to disconnect will hang if this code is not commented out

If you run this and it disconnects fine, then it may be the vSphere version - we're currently on 6.0 and due to upgrade next month to 6.7U1

Code attached.

Reply
0 Kudos
LucD
Leadership
Leadership

I don't have a real explanation of the why, but when you take the Disconnect-VIServer out of the Exit button event it all seems to work.

It looks as if a Windows forms event handler method execute in a special context.

When you change the last part to the following it works.

# Exit and close form

$ExitButton.Add_Click( {

   $script:CanClose = $true

   $form.close()

   })

# Show form

$form.ShowDialog() | Out-Null

$form.Dispose()

Disconnect-VIServer -Server * -Force -Confirm:$false


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Luc

Many thanks, yes that's exactly my experience too.

As mentioned, I'm going to upgrade to 6.7U1 soon - think it's worth raising it as an issue with VMware once I'm up to date (or feel free to do this yourself)?

There seems to be something specifically about the Get-Tag / Get-TagAssignment operations that cuases this - it doesn't happen with Get-VM or any other command that I've found.

Reply
0 Kudos
LucD
Leadership
Leadership

Not too sure to be honest.
They will probably point at Windows Forms as the culprit.
Although it looks like a combination of the current Tag cmdlets and Windows Forms imho.

You didn't mention which PowerCLI version you are using.
Since one of the previous versions all the Tag cmdlets are using the REST API.

That could make a difference.

Before eventually upgrading your PowerCLI version, it could be interesting is to try the same with the functions from the VMware.Community.CISTag module (which also uses the REST API).

See New Community Module for Tag Management


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Luc

No idea if you'll revisit this thread, but it seems the issue with tag Commandlets and PS/Forms is a bit wider:

This operation will immediately hang the Windows Form (initiated from a button click).

   $CurrentVM | New-TagAssignment -Tag $RequiredTag -ErrorAction Stop

I've attached my latest version of the full script including the functionality to change a tag in case you want to confirm this

Reply
0 Kudos
LucD
Leadership
Leadership

Which PowerCLI version are you using?
Did you try the community module (all REST API based)?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Currently on

VMware.PowerCLI 11.2.0.12780525

Not tried the community module as I was trying to avoid another dependency, but I'll take a look at it now.

Reply
0 Kudos
LucD
Leadership
Leadership

Since you are on the latest PowerCLI module, those Tag related cmdlets are already using the REST API.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Ok thanks.

To note - although the tag assignment commandlet hangs the windows form - it does actually change the tag before that happens.

Reply
0 Kudos
LucD
Leadership
Leadership

I wonder if the script does nothing for a certain period (perhaps a sleep for a couple of minutes), if that would also freeze the form?

You could also consider firing of the query for Tags in a background job (with Start-Job).
But since you noticed that the Get-TagAssignment is possibly not the cause of the freeze, I'm wondering if it would make a difference.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Get-TagAssignment doesn't cause any issues until Disconnect-VIServer is run and for this, the workaround to disconnect after the form closes is fine, but Set-TagAssignment immediately crashes the form and thus prevents me from coding that functionality at all.

I may give Start-Job a try but as you say this might just end up with hung jobs instead

I've raised a support request with low priority to report the issue.

I've also noticed that removing a tag (Remove-TagAssignment) works ok.

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

Ok for info, the tag Commandlets work fine if they're started in a job instead, so it looks like there's some specific error with calling the commandlet from a Windows Form.

Function SetTag ($InputVars) {

   Connect-VIServer $InputVars.VIServer

   $RequiredTag = Get-Tag -Category $InputVars.TagCategory -Name $InputVars.Tag

   Get-VM $InputVars.VM | New-TagAssignment -Tag $RequiredTag

}

# Export function to string so that it can be called in jobs as an initialization script

$ExportFunction = [scriptblock]::Create(@"

Function Set-Tag {

$function:SetTag

}

"@)


$InputVars = [pscustomobject] @{VIServer = $global:DefaultVIServer.Name;VM = $CurrentVM;TagCategory = $BackupTagCategory;Tag = $SelectPolicyCombo.Text}

$TagJob = Start-Job -Name $CurrentVM -ScriptBlock {Set-Tag $args[0]} -ArgumentList $InputVars -InitializationScript $ExportFunction -ErrorAction Stop

It's a bit clunky as you have to pass everything to the job, including info to start a new VIServer connection, but it does work ok.

Reply
0 Kudos
LucD
Leadership
Leadership

I suspect there might be some interference in the registration and handling of events.
A Windows Form needs to register the code snippets for all events that can happen on or be triggered from a form (button click, text field edit...).
I suspect the Tag cmdlets one way or the other seem to influence/impact the form event-sink.

By running the Tag cmdlets in a background job, they actually run in a separate process.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

For info, I reported this to VMware and have now received notice that the Dev team have acknowledged the fault.

They have scheduled work on fixing this to start end June 2019.

Reply
0 Kudos