Reboot a VM and wait until tools are running before rebooting the next vm

[joshuahlrowe]

I'm trying to write a script that will reboot a vm and wait until the vm has tools running before the next vm starts the reboot process. Also the reason I used $toolsStatus -ne "toolsNotRunning" instead of $toolsStatus -eq "toolsOK" is because we have a lot of tools that are not current. The reason I want to do this is because I want a script to reboot my Domain Controllers and I don't want the second DC to reboot until the first DC is back up and running. Here is what i have currently however when I run the script the first VM reboots and within 5 seconds the second vm starts to reboot before the first vm is back up and running. Any help would be appreciated.

#Connect to the vCenter Server

  Connect-VIServer vcenter.domain.local

#Reboot servers in proper order and wait for tools to be running until the next vm reboots

  $Input = "VMTEST1","VMTEST2"

  foreach ($VM in $input){

  Restart-VM -VM $VM -Confirm:$false

  Do {$toolsStatus = (Get-VM $VM).extensiondata.Guest.ToolsStatus

  write-host $toolsStatus

  sleep 5} until ( $toolsStatus -ne "toolsNotRunning" )}

Thanks,

Josh

[LucD]

What is the actual content of the ToolsStatus property is?

What does the Write-Host show?

[joshuahlrowe]

I think what the issue is, is that it's not waiting long enough to check the tools status initially. The past few times I've ran the script on the first vm it shows toolsok and then goes to the next vm. This time when I ran the script I got the following output before it went to the second vm:

toolsNotRunning

toolsNotRunning

toolsNotRunning

toolsNotRunning

toolsNotRunning

toolsNotRunning

toolsNotRunning

toolsOk

However when it got to the second vm it immediately said:

toolsOk

Maybe there is a way to wait 5 seconds or something before the initial tools check of the vm? I'm open to suggestions.

[LucD]

You try to place a pause between the Restart-VM and the Do-Until loop.

I suspect it takes a bit before the restart is actually reflected in the status of the tools.

The most optimal way would be to wait for the VmPoweredOffEvent event after the Restart-VM

[joshuahlrowe]

I tried the following however I'm getting the same results as without the powerstate line. What command would I use to wait for vmpoweroffevent?

#Connect to the vCenter Server

  Connect-VIServer vcenter.domain.local

#Reboot servers in proper order and wait for tools to be running until the next vm reboots

  $Input = "VMTEST1","VMTEST2"

  foreach ($VM in $input){

  Restart-VM -VM $VM -Confirm:$false

  while ((Get-VM $VM).PowerState -eq "PoweredOff") { Start-Sleep -Seconds 5 }

  Do {$toolsStatus = (Get-VM $VM).extensiondata.Guest.ToolsStatus

  write-host $toolsStatus

  sleep 5} until ( $toolsStatus -ne "toolsNotRunning" )}

[jpsider]

Why not use the 'wait-tools' cmdlet?

#Connect to the vCenter Server
 Connect-VIServer vcenter.domain.local
#Reboot servers in proper order and wait for tools to be running until the next vm reboots
 $Input = "VMTEST1","VMTEST2"
 foreach ($VM in $input){
  Restart-VM -VM $VM -Confirm:$false
  #or Restart-VMGuest -vm $vm -Confirm:$false
  start-sleep -s 3
  wait-tools
}

[joshuahlrowe]

I tried that and I get the following:

wait-tools : 12/27/2016 8:27:06 AM      Wait-Tools              Value cannot be found for the mandatory parameter VM

At C:\Scripts\RebootServers.ps1:9 char:3

+   wait-tools

+   ~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Wait-Tools], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.WaitTools

[jpsider · Accepted solution]

Whoops, forgot to pass in the $vm

Give this a shot

#Connect to the vCenter Server

Connect-VIServer vcenter.domain.local

#Reboot servers in proper order and wait for tools to be running until the next vm reboots

$Input = "VMTEST1","VMTEST2"

foreach ($VM in $input){

  Restart-VM -VM $VM -Confirm:$false

  #or Restart-VMGuest -vm $vm -Confirm:$false

  start-sleep -s 3

  wait-tools -vm $VM

}

[jpsider]

and I just looked at the help file..   get-help -full wait-tools

You might be able to just do this:

#Connect to the vCenter Server

Connect-VIServer vcenter.domain.local

#Reboot servers in proper order and wait for tools to be running until the next vm reboots

$Input = "VMTEST1","VMTEST2"

foreach ($VM in $input){

  Restart-VMGuest $vm | Wait-Tools

}

[joshuahlrowe]

This worked perfectly! Thank you! I tried the shorter command and it rebooted both VMs without waiting for tools.

[jpsider]

ok, the longer version it is! Glad I could help.

[elargent]

Can you please provide the longer version of the Wait-Tools command you are using. 

[LucD]

I suspect the user was alluding to Restart-VM vs Restart-VMGuest.
Where the latter one would be the "long" version.

[elargent]

That is what I thought also, here is what I am trying to run. The second VM starts the reboot process just a few seconds after the first one. Any thoughts?

Connect-VIServer servername.domain

$Input = "server1" , "server2"

foreach ($VM in $input) {

Restart-VMGuest -VM $VM | Wait-Tools

}

[LucD]

You might be running those VMs on super-fast HW :smileygrin:

Seriously though, I hardly ever use Wait-Tools anymore (since similar experiences).

I discovered that, for me at least, it is a lot better to wait until guestOperationsReady becomes $true.

See for example lines 68-72 in my Deploy Photon 2.0 – Part 2 post.

Only when that property turns to $true can you start using Invoke-VBScript for example.

The VMware Tools might already be running some time before that occurrence.

[vespavbb]

I tried that, but it is very unreliable. Sometimes it works, sometimes not

Restart-VMGuest -VM $VM | Wait-Tools

Even this is not working sometime. The VM is off and the status comes back with guestToolsRunning

Restart-VMGuest -VM $VM -Confirm:$false | out-null

sleep 1

Write-Host “Waiting....reboot $VM”

sleep -Seconds 1

do {

Set-Variable -name ToolsStatus -Value (Get-VM $VM).extensiondata.Guest.ToolsRunningStatus

Write-Host $toolsStatus

sleep 5

}

until ($toolsStatus -eq "guestToolsRunning")

[LucD]

Since you seem to address this to me, I already replied in the other thread, that this is not working all the time.

What is the problem with using the GuestOperationsReady property as I mentioned here and in the other thread?

[vespavbb]

Thank you, I really appreciate your patience.
But I really do not understand why the simple loop does not work, the trick with the toolsStatus would be nice

I will check your other solution..

[LucD]

Because Restart-VMGuest triggers the restart inside the guest OS and then the cmdlet comes back.

When the cmdlet comes back, the VMware Tools are most probably still running.

And the loop will be run once and the script continues.

I think I mentioned that you first have to wait for the poweredOff state.

[geeky_kumar]

Give this code a shot. It works perfectly for me. This leverages GuestOperationsReady property as suggested by LucD.

There is one issue with GuestOperationsReady property. It becomes 'true' when vmwaretools service start. However during windows boot process services can start even though the other stuff like GPO are still loading in the back ground. Meaning the status changes even though  server boot process is not yet completed (Login screen displayed)

I have tested Wait-Tools command with timeout, even it prematurely declares the boot process as compleed.

To check whether the boot process completed  is to check for "lsass" process is running. This is the process responsible for the logon of the user and which presents the Ctrl Alt screen. To check the process I used Invoke-VMScript command. If the process starts the while loop ends executing. The while loop time out eventually if any of the boot process conditions are NOT meet.

Make sure to change the $localuser and $localpwd to local admin account values as per your environment.

---Code--

$Run = [System.Convert]::ToBoolean($True)
$scripttxt = "(Get-Process -Name lsass).ProcessName"
$Timer = [Diagnostics.Stopwatch]::StartNew()
While ($run) {

Start-Sleep -Seconds 5

#If guest operations ready
If ((Get-VM $vmname).ExtensionData.Guest.GuestOperationsReady) {

#Check if lsass process started (logon screen)
$output = $((Invoke-VMScript -ScriptText $scripttxt -VM $vmname -ScriptType Powershell -GuestUser $localuser -GuestPassword $localpwd -ErrorAction SilentlyContinue).ScriptOutput)
If ($output -like "*lsass*"){

Write-Host "Server presenting Ctrl Alt Screen"
$timer.Stop()
Write-Host "Reboot completed."
$Run = [System.Convert]::ToBoolean($False)
break
}

}

## The action timed out.
If ($timer.Elapsed.TotalSeconds -gt 300){

$timer.Stop()
Write-Host "Timeout occured"
$Run = [System.Convert]::ToBoolean($False)
Exit
}

}