I know variations of this question have been asked more than once, and after looking at different threads I've found pieces of my answers but not a complete script. I'm pretty new to scripting so bare with me please
What I'm looking to do is move away from the APC PowerChute Network Shutdown on all VM's/Hosts and possibly move to a scripted shutdown with powershell. What I've been trying to piece together is a script tha:
1) gracefully shuts down all VM's
Wait timer for all VM's to shutdown
2) Hard shutdown of certain VM's that can't gracefully shut down
3) puts the hosts into maintenance mode
wait timer to move into maintenance mode
4) shuts down the ESX hosts
Any direction of how I can do this (if it is possible?) or references where I should be looking would be extremely helpful.
Kyle
There is one catch to such a script, the Shutdown-VMGuest nor the ShutdownGuest method from the SDK return a status.
So it is difficult to determine when a guest will not shutdown the soft way.
A quick-and-dirty solution is to wait a predefined number of seconds.
Anyone know a better solution for this problem ?
There are currently no cmdlets for putting an ESX server in maintenance mode or for shutting down an ESX server.
Hence the use of the EnterMaintenanceMode_Task and ShutdownHost_Task methods from the SDK.
An alternative would be to use the VITK Extensions for this but the Extensions require PowerShell v2 CTP3 which might not be admissible in every shop.
$MyHost = <ESX-server> $timeout = 0 $evacuatePoweredOffVms = $false $force = $false $WaitSeconds = 300 $esx = Get-VMHost $MyHost | Get-View # Shutdown guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest # Wait sleep $WaitSeconds # Hard poweroff the remaining guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Stop-VM -Confirm:$false # Enter maintenance mode if(-not $esx.runtime.inMaintenanceMode){ $taskMoRef = $esx.EnterMaintenanceMode_Task($timeout,$evacuatePoweredOffVm) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef} } # Shutdown $taskMoRef = $esx.ShutdownHost_Task($force) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef}
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
There is one catch to such a script, the Shutdown-VMGuest nor the ShutdownGuest method from the SDK return a status.
So it is difficult to determine when a guest will not shutdown the soft way.
A quick-and-dirty solution is to wait a predefined number of seconds.
Anyone know a better solution for this problem ?
There are currently no cmdlets for putting an ESX server in maintenance mode or for shutting down an ESX server.
Hence the use of the EnterMaintenanceMode_Task and ShutdownHost_Task methods from the SDK.
An alternative would be to use the VITK Extensions for this but the Extensions require PowerShell v2 CTP3 which might not be admissible in every shop.
$MyHost = <ESX-server> $timeout = 0 $evacuatePoweredOffVms = $false $force = $false $WaitSeconds = 300 $esx = Get-VMHost $MyHost | Get-View # Shutdown guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest # Wait sleep $WaitSeconds # Hard poweroff the remaining guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Stop-VM -Confirm:$false # Enter maintenance mode if(-not $esx.runtime.inMaintenanceMode){ $taskMoRef = $esx.EnterMaintenanceMode_Task($timeout,$evacuatePoweredOffVm) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef} } # Shutdown $taskMoRef = $esx.ShutdownHost_Task($force) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef}
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Thanks LucD
This gives me a great base to go off of. I noticed the same thing about not really returning anything when its shutdown but the timer should be just fine as I know about how long it takes for them to shut down. Our APC has about 30-40min of runtime so I have a lot of give when it comes to how fast it shuts down. I'll try what you posted and open up another thread if needed later on down the road.
Thanks again
Kyle
Is there a reason you want the host in maintenace mode? I haven't tried it lately with ESX, but with ESXi the host comes back up in maintenance mode and you have to then change that status.
If you have DRS in automatic mode, the switch to maintenance mode will vmotion all your running guests to other servers.
And yes, in ESX 3.5u3 it comes back in maintenance mode as well.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
The main reason I was thinking about putting it in maintenance mode is that I figured it was a fail-safe to make sure everything was shutdown before killing the ESX host. If the APC device triggered a network shutdown, someone would be coming into the building to check out a) why there was a power failure and/or why the generator ran out of gas ... or b) why the server room turned into an oven
So them coming back into maintenance mode isn't that big of an issue for me considering we only have 4 hosts. We would power the VM's on ourselves manually anyways.
Also wouldn't the command "Get-VMHost $myhost | Set-VMhost -state mainenance" work as well?
Kyle
Correct, that brings it to maintenance mode as well.
I'll update my script.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
For the $WaitSeconds = 300 variable, I can define like WaitSecondsLong = 300 and WaitSecondsShort = 120 or along those lines? Or does the sleep command look for a WaitMinutes/Seconds or whatever?
Also the script is for a specific host, If I wanted to do a total network shutdown of all of our hosts, which is only 4, should I just eliminate the Get-VMhost $myhost from the script or should I do each section and have$myhost1, $myhost2, etc..
I wasn't sure if I do the Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest if that would be too much for VirtualCenter to process or would it schedule all the shutdowns and just pound out a couple at a time and queue up the rest?
Kyle
I'm not sure I understand your question about the sleep cmdlet.
With some changes the script could be changed to stop all VMHosts.
Best would perhaps be to create a function that takes as a parameter the VMHostname.
You could then pipe the names of all your VMHosts to the function.
Something like this:
function Shutdown-VMHost{ param($MyHost) $timeout = 0 $evacuatePoweredOffVms = $false $force = $false $WaitSeconds = 300 $esx = Get-VMHost $MyHost | Get-View # Shutdown guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest # Wait sleep $WaitSeconds # Hard poweroff the remaining guests Get-VMHost $MyHost | Get-VM | Where-Object {$_.PowerState -eq "PoweredOn"} | Stop-VM -Confirm:$false # Enter maintenance mode if(-not $esx.runtime.inMaintenanceMode){ $taskMoRef = $esx.EnterMaintenanceMode_Task($timeout,$evacuatePoweredOffVm) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef} } # Shutdown $taskMoRef = $esx.ShutdownHost_Task($force) $task = Get-View $taskMoRef while($task.info.state -eq "running" -or $task.info.state -eq "queued"){$task = Get-View $taskMoRef} } Get-VMHost | %{Shutdown-VMHost $_.Name}
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
This is the script I've been working on based off of this example.
I've added "tiering" of the VMs to allow for an orderly shutdown based
on dependencies. I accomplished this by creating a custom field "Tier" and assigning it a
numeric value between 0 and 3 depending on the role of the VM. VMs in
Tier 0 get shutdown immediately and only when all VMs in that tier are
powered off does the script move on to shutting down the next tier. We
use PRTG for monitoring so I've used the web API they provide to pause the top level node before beginning shutdown of the system.
I've attached my script, but I couldn't figure out how to insert a script block like you did above.
Great script, very inventive use of the custom attribute.
PS: in the editor under the Plain Text tab, there is a button marked "Plain Text Markup Help". This will show all the markup codes.
For the box I used, use the block (without the spaces).
____________
Blog: LucD notes
Twitter: lucd22
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference