cabraun
Enthusiast
Enthusiast

Scripting a mass shutdown in order

Jump to solution

I have been asked to script a method of shutting down all servers (Physical & Virtual) in our datacenter in as quick a way as possible but in a particular order.  With regards to the VMs I already shutdown many over the weekends with a script that runs get-vm "<my server name>" | shutdown-vmguest -confirm:$false against a list of a dozen or so servers.  But that has been the extent of my PowerCLI scripting to date.  And those VMs never change in that list.

Can anyone maybe point me in the right direction on getting started with the following

Nightly run a script that will gather the list of all VMs in my environment (to capture any newly created or recently deleted VMs) and put all existing VMs into a CSV or something in an order based on test in the notes section so that for example all test/dev application servers are grouped together to be shutdown before test/dev database servers.

Then have a script just waiting that, when executed will begin shutting down all the VMs in the most recent CSV in the proper order.

So the reason for this, if you are interested is that 2 weeks ago we had a power outage and we have no generator.  Our UPS allows the datacenter to run for about 30 minutes.  Of course since these things can only ever happen at night, by the time someone got here to start shutting things down they only had a few minutes to turn off key physical and virutal machines and they of course ran out of time.  So now we are looking for a logon and perform a 1 click solution to start shutting down everything.

We will be having a scheduled power outage next week to replace some electical equipment and would like to have the script ready by then.

Anyone willing to lend me a hand or point me in the right direction?

Thanks

0 Kudos
1 Solution

Accepted Solutions
ae_jenkins
Enthusiast
Enthusiast

Hi

Nothing really wrong with the approach, but why bother with CSV if you can just do it hot?

I'm assuming you something in the notes that indicate what it is (testDB, prodDB, testApp).  You can also do this if you have a server naming convention that indicates what it is.

$folder = get-folder "White"

get-vm | where {$_.notes -like '*testdb*'} | move-vm -destination $folder


Obviously, I'm guessing at your "notes" conventions and your folder convention.  If the VM already exists in the folder I think it just skips it without throwing an error, but you definately should test.  Use the '-whatif' switch at the end of the second line to see if it throws errors.

I'm thinking you do this like once a week to keep your environment 'clean' to prep for a power-outage ahead of time

View solution in original post

0 Kudos
7 Replies
AureusStone
Expert
Expert

I think the best solution would be to create a script that shuts your environment down in order without a csv file.

Get-VM | Where {$_.Notes -eq "Test" -and $_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest -Confirm:$false

Get-VM | Where {$_.Notes -eq "Production" -and $_.PowerState -eq "PoweredOn"} | Shutdown-VMGuest -Confirm:$false

The -eq command will only match exactly, but it case insensitive.  It would be better to organise your VMs in to folders and shutdown based on the folder the VMs are in.  You don't want a typo to break your script.

Is the 30 minute window long enough to shutdown all of your VMs.  If not you can run shutdown /i from windows or something else from Linux.  Otherwise if you do not have network connectivity you can use PowerCLI\VIX to run shutdown scripts or all of your servers.

ae_jenkins
Enthusiast
Enthusiast

To comment on top of Mr. Stone, it would be cleaner to organize your testApp/testDB servers in folders for shutdown.  You could have a script that periodically checks for VMs not in your folder structure and move it according to the notes.  That being said, here is another example:

$vm = get-vm | where {$_.Powerstate -eq "PoweredOn"} | select name, powerstate, notes | sort notes, name

$vm | export-csv -path c:\temp\output.csv -notypeinformation


If you had it set up with folders, you could combine the shutdown guest cleanly like Mr. Stone has

get-folder "Test Applications" | get-vm | where {$_.Powerstate -eq "PoweredOn} | shutdown-vmguest -confirm:$false

start-sleep 60

get-folder "Test Database" | get-vm | where {$_.Powerstate -eq "PoweredOn} | shutdown-vmguest -confirm:$false

0 Kudos
ae_jenkins
Enthusiast
Enthusiast

Darn formatting

First example:

$vm = get-vm | where {$_.Powerstate -eq "PoweredOn"} | select name, powerstate, notes | sort notes, name

$vm | export-csv -path c:\temp\output.csv -notypeinformation


2nd example

get-folder "Test Applications" | get-vm | where {$_.Powerstate -eq "PoweredOn} | shutdown-vmguest -confirm:$false

start-sleep 60

get-folder "Test Database" | get-vm | where {$_.Powerstate -eq "PoweredOn} | shutdown-vmguest -confirm:$false

0 Kudos
ae_jenkins
Enthusiast
Enthusiast

I give up.  Formatting is kicking me in the head this morning.   I thought there used to be a "preview" button before

0 Kudos
cabraun
Enthusiast
Enthusiast

I like the shutdown by folder idea and I already have the vast majority of my VMs grouped in folders that would be appropriate for the task.  We actually patch our servers by color codes based on the servers role in the environment so, for example, I have folders called "White", "Yellow", "Green", "Blue", "Orange", "Red" and "Black" and each VM is in the appropriate folder based on the order it should be patched and rebooted.

So if I wanted to shutdown all the VMs in the "White" folder followed by the "Yellow" folder etc it would be:
=====================================
=====================================
#Choose Source vCenter
$sourceVI = Read-Host "Please enter the name of the vCenter Server";
#Enter Credentials
$creds = get-credential
#Choose Datacenter (Prod or Test)
$datacenter = Read-Host "Please give the name of the datacenter you would like to run against"
#Connect to Source vCenter
connect-viserver -server $sourceVI -credential $creds
#Get List of VMs in Datacenter
$vmlist = get-datacenter $datacenter -Server $sourceVI| get-vm
#Get Information in VM Notes
$Report =@()
    foreach ($vm in $vmlist) {
        $row = "" | Select Name, Notes
        $row.name = $vm.Name
        $row.Notes = $vm | select -ExpandProperty Notes
        $Report += $row
    }
#Complie Information Into CSV
$report | Export-Csv "d:\scripts\vms-with-notes-$($datacenter).csv" -NoTypeInformation
=====================================
=====================================
OK, so this is where I am presently stuck.  How would I then take the information in the csv and use it to move VMs into the correct folder based on something in the notes like "Patching Color = White" or "Patching Color = Yellow" etc to move it into the White or Yellow folder etc?  I know somewhere along the way there will be a "Move-VM -VM servername -Destination White" command but first I need to identify which servers are not in their correct folder.
=====================================
=====================================
#
#Begin Shutdown of Running VMs with 60 pause between colors
#
get-folder White | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Yellow | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Green | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Blue | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Orange | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Red | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
start-sleep 60
get-folder Black | get-vm | where {$_.Powerstate -eq PoweredOn} | shutdown-vmguest -confirm:$false
=====================================
=====================================
Or maybe I am all wrong especially on the top portion.
0 Kudos
ae_jenkins
Enthusiast
Enthusiast

Hi

Nothing really wrong with the approach, but why bother with CSV if you can just do it hot?

I'm assuming you something in the notes that indicate what it is (testDB, prodDB, testApp).  You can also do this if you have a server naming convention that indicates what it is.

$folder = get-folder "White"

get-vm | where {$_.notes -like '*testdb*'} | move-vm -destination $folder


Obviously, I'm guessing at your "notes" conventions and your folder convention.  If the VM already exists in the folder I think it just skips it without throwing an error, but you definately should test.  Use the '-whatif' switch at the end of the second line to see if it throws errors.

I'm thinking you do this like once a week to keep your environment 'clean' to prep for a power-outage ahead of time

View solution in original post

0 Kudos
cabraun
Enthusiast
Enthusiast

Thanks.  I want to keep it as simple as possible.  Someone here mentioned the csv option so my mind was set on that, but your proposal makes much more sense.

So here is what I have tested:

I took a test VM that was off and I started it and put it into a folder other than "Test" and I added a note of 12345678 to it.  I then ran the following script:

==================

==================

#Choose Source vCenter
$sourceVI = Read-Host "Please enter the name of the vCenter Server";
#Enter Credentials
$creds = get-credential
#Choose Datacenter (Prod or Test)
$datacenter = Read-Host "Please give the name of the datacenter you would like to run against"
#Connect to Source vCenter
connect-viserver -server $sourceVI -credential $creds
$vmlist = get-datacenter $datacenter -Server $sourceVI| get-vm
$folder = get-folder "Test"
get-vm | where {$_.notes -like '*12345678*'} | move-vm -destination $folder
get-folder test | get-vm | where {$_.Powerstate -eq "PoweredOn"} | shutdown-vmguest -confirm:$false
start-sleep 60

==================

==================

As expected my VM was moved and then shutdown.

So now I just need to add the correct folder names and notes that are appropriate to look for and I should be good to go.

Thanks for your help.

0 Kudos