Hi guys, just implemented Vxrail/Vsan six nodes stretched between two sites.
Our UPS doesn't support Vsan, only shutting down the VM's and hosts in case of power failure, alternative can launch a script.
Better then nothing, but in case of a power failure in one site with UPS software configured the result is half infrastructure down.
Without UPS software, HA reboot the VM's in the second site (uncontrolled) and Hardware power down abnormaly.
The idea is to make two scripts to launch specifically in SIte A and B:
1) Disable HA.
2) Setting DRS in manual mode.
3) Move VM from host esx4 to host esx1, then from esx5 to esx2 and finally to esx6 to esx3.(and a second script vice-versa)
4) Put hosts esx4 esx5 and esx6 in maintenance mode with "ensure accessibility".
5) Shutting down esx4,esx5 and esx6.
6) Enable DRS.
The same script will be usable also to plan a controlled DR test.
I'm not a "black belt" script man:)
Thanks for any help and greetings from Italy!
@LucDthank you for the tips, after further research I've create the script and it's working, two questions if you can help to improve the result:
1) Is it possible to filter the "vCLS" VM's to they don't move?
2) I received this warning during the script execution: "WARNING: The 'state' property of VMHost type is deprecated. USe the 'ConnectionState' property instead.
Thanks!
param(
[string]$Server = "vcenter",
[string]$Username = "vcenteruser",
[string]$Password = "vcenterpwd",
[string]$clusterName = "Clustername"
)
$conn = Connect-VIServer $Server -Username $Username -Password $Password -Force
set-cluster $clusterName -HAEnabled $false -Confirm:$false
set-cluster $clusterName -drsautomationlevel manual -confirm:$false
Get-VM -Location esx4 | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5 | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Move-VM -Destination (Get-Vmhost esx1)
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Set-VMHost esx5 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Set-VMHost esx6 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Stop-VMHost esx4 -Confirm:$false
Stop-VMHost esx5 -Confirm:$false
Stop-VMHost esx6 -Confirm:$false
set-cluster $clusterName -drsautomationlevel fullyautomated -confirm:$false
set-cluster $clusterName -HAEnabled $true -Confirm:$false
Disconnect-VIServer $conn -Force -Confirm:$false
You can exclude the vCLS VMs with a Where-clause
Get-VM -Location esx4 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx1)
That is just a warning, until that parameter is actually dropped in favour of ConnectionState, you can keep using State.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
There aren't any "special" tricks involved as far as I can see.
1) and 2) can be done with the Set-Cluster cmdlet, respectively the HAEnabled and the DrsAutomationLevel parameters.
3) is done with the Move-VM cmdlet and the Destination parameter.
4) Is with the Set-VMHost and the State parameter.
5) The Stop-VMHost cmdlet
6) See 2) above.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
@LucDthank you for the tips, after further research I've create the script and it's working, two questions if you can help to improve the result:
1) Is it possible to filter the "vCLS" VM's to they don't move?
2) I received this warning during the script execution: "WARNING: The 'state' property of VMHost type is deprecated. USe the 'ConnectionState' property instead.
Thanks!
param(
[string]$Server = "vcenter",
[string]$Username = "vcenteruser",
[string]$Password = "vcenterpwd",
[string]$clusterName = "Clustername"
)
$conn = Connect-VIServer $Server -Username $Username -Password $Password -Force
set-cluster $clusterName -HAEnabled $false -Confirm:$false
set-cluster $clusterName -drsautomationlevel manual -confirm:$false
Get-VM -Location esx4 | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5 | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Move-VM -Destination (Get-Vmhost esx1)
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Set-VMHost esx5 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Set-VMHost esx6 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
Stop-VMHost esx4 -Confirm:$false
Stop-VMHost esx5 -Confirm:$false
Stop-VMHost esx6 -Confirm:$false
set-cluster $clusterName -drsautomationlevel fullyautomated -confirm:$false
set-cluster $clusterName -HAEnabled $true -Confirm:$false
Disconnect-VIServer $conn -Force -Confirm:$false
You can exclude the vCLS VMs with a Where-clause
Get-VM -Location esx4 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Where{$_.Name -notmatch '^vCLS'} | Move-VM -Destination (Get-Vmhost esx1)
That is just a warning, until that parameter is actually dropped in favour of ConnectionState, you can keep using State.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
@LucDthank you, working great!
The last question, is it possible to write a log execution file?
Have a good day
You can add one or more lines containing the Out-File cmdlet.
It allows you send text to an external file.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Hi @LucD ,
during the tests one of three node don't enter in maintenance node before shutting down.
Actually I've implemented some "sleeps" and two cycle of the maintenance mode command.
Is it possible to check the maintenance mode state after send the command and if maintenance mode isn't reach send the command more times?
Thank you in adavnce and happy Easter!
param(
$Server = "vcenter",
$Username = "user",
$Password = "pwd",
$clusterName = "Clustername",
$VMexclusionName = "^vCLS",
$LogPath = "C:\log.log"
)
Start-Transcript -Path $LogPath
$conn = Connect-VIServer $Server -Username $Username -Password $Password -Force
set-cluster $clusterName -HAEnabled $false -Confirm:$false
set-cluster $clusterName -drsautomationlevel manual -confirm:$false
Get-VM -Location esx4 | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5t | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx1t)
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Set-VMHost esx5 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Set-VMHost esx6 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Set-VMHost esx5 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Set-VMHost esx6 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
sleep 5
Stop-VMHost esx4 -Confirm:$false
Stop-VMHost esx5 -Confirm:$false
Stop-VMHost esx6 -Confirm:$false
set-cluster $clusterName -drsautomationlevel fullyautomated -confirm:$false
set-cluster $clusterName -HAEnabled $true -Confirm:$false
Disconnect-VIServer $conn -Force -Confirm:$false
Stop-Transcript
You can check the State by doing a Get-VMHost.
I don't think that sending the command multiple times will help.
To make sure your command for Maintenance Mode is accepted by the ESXi node, you can also check the vents for an EnteringMaintenanceModeEvent
That way you know the command was received/accepted by the ESXi node.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
@LucDthank you for the reply, I ask you the last favour, please, can you write the necessary commands?
Thank you
You can replace the sleep with a While-block.
Something like this
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost esx4).State -ne 'Maintenance'){
sleep 5
}
When one of the nodes fails to go into maintenance mode, you might end up in an endless loop.
You could use a timeout for that.
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost esx4).State -ne 'Maintenance' -and $interval -le $timeOut){
sleep 5
$interval = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
}
if($interval -gt $timeOut){
Write-Host "Timeout"
exit
}
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Thank you again @LucD I've just replace the set-maintenace and sleep rows only with your second "block" of commands, is it correct?
I've try and seems to working good, below the entire script:
param(
$Server = "vcenter",
$Username = "user",
$Password = "pwd",
$clusterName = "Clustername",
$VMexclusionName = "^vCLS",
$LogPath = "C:\log.log"
)
Start-Transcript -Path $LogPath
$conn = Connect-VIServer $Server -Username $Username -Password $Password -Force
set-cluster $clusterName -HAEnabled $false -Confirm:$false
set-cluster $clusterName -drsautomationlevel manual -confirm:$false
Get-VM -Location esx4 | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx3)
Get-VM -Location esx5t | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx2)
Get-VM -Location esx6 | Where{$_.Name -notmatch $VMexclusionName} | Move-VM -Destination (Get-Vmhost esx1t)
Set-VMHost esx4 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost esx4).State -ne 'Maintenance' -and $interval -le $timeOut){
sleep 5
$interval = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
}
if($interval -gt $timeOut){
Write-Host "Timeout"
exit
}
Set-VMHost esx5 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost esx5).State -ne 'Maintenance' -and $interval -le $timeOut){
sleep 5
$interval = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
}
if($interval -gt $timeOut){
Write-Host "Timeout"
exit
}
Set-VMHost esx6 -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost esx6).State -ne 'Maintenance' -and $interval -le $timeOut){
sleep 5
$interval = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
}
if($interval -gt $timeOut){
Write-Host "Timeout"
exit
}
Stop-VMHost esx4 -Confirm:$false
Stop-VMHost esx5 -Confirm:$false
Stop-VMHost esx6 -Confirm:$false
set-cluster $clusterName -drsautomationlevel fullyautomated -confirm:$false
set-cluster $clusterName -HAEnabled $true -Confirm:$false
Disconnect-VIServer $conn -Force -Confirm:$false
Stop-Transcript
Yes, that is correct.
You could store all ESXi names in an array and then loop through that array.
That way you avoid having that part x-times in your script
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Great!! Can I send you a beer!?:)
I've no idea how to collect the esxi array names, if you want to give another help write the optimization, but is just "cosmetic", right?
You could do something like this
param(
$Server = "vcenter",
$Username = "user",
$Password = "pwd",
$clusterName = "Clustername",
$VMexclusionName = "^vCLS",
$LogPath = "C:\log.log"
)
Start-Transcript -Path $LogPath
$conn = Connect-VIServer $Server -Username $Username -Password $Password -Force
Set-Cluster $clusterName -HAEnabled $false -Confirm:$false
Set-Cluster $clusterName -DrsAutomationLevel manual -Confirm:$false
Get-VM -Location esx4 | where { $_.Name -notmatch $VMexclusionName } | Move-VM -Destination (Get-VMHost esx3)
Get-VM -Location esx5 | where { $_.Name -notmatch $VMexclusionName } | Move-VM -Destination (Get-VMHost esx2)
Get-VM -Location esx6 | where { $_.Name -notmatch $VMexclusionName } | Move-VM -Destination (Get-VMHost esx1t)
$esxNames = 'esx4', 'esx5', 'esx6'
foreach ($esx in $esxName) {
Set-VMHost $esx -State Maintenance -VsanDataMigrationMode EnsureAccessibility -Confirm:$false
while ((Get-VMHost $esx).State -ne 'Maintenance' -and $interval -le $timeOut) {
sleep 5
$interval = (New-TimeSpan -Start $start -End (Get-Date)).TotalSeconds
}
if ($interval -gt $timeOut) {
Write-Host "Timeout"
exit
}
Stop-VMHost $esx -Confirm:$false
}
Set-Cluster $clusterName -DrsAutomationLevel fullyautomated -Confirm:$false
Set-Cluster $clusterName -HAEnabled $true -Confirm:$false
Disconnect-VIServer $conn -Force -Confirm:$false
Stop-Transcript
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference