VMware Cloud Community
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

i am trying to do the datastore balance using powercli (Storage DRS in powercliscript), can you please someone help me on this, if it is already thre in infra

i am trying to do the datastore balance using powercli (Storage DRS in powercliscript), can you please someone help me on this, if it is already thre in infra

Regards, Satheesh
1 Solution

Accepted Solutions
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

Thanks a lot Brother, i was able to complete this with the below codes(ofcourse with your input too)

do{
$i
$VMtomove = $VMslist["$i"]
$VMstoMove += $VMtomove
$VMdisksizeperDS = ($VMtomove |Get-HardDisk |?{$_.Filename -match $lowDS.Name}|foreach{$_.CapacityGB} |Measure-Object -Sum).sum
 
$Datastores = $VMHosts | Get-Datastore | where {($_.Name -notlike $DNIDSNames) -and ($_.Name -notlike $DNIDSNames1 ) -and ($_.Name -notlike $DNIDSNames2)} 
 
$VMSize = (($VMtomove |Get-HardDisk| Foreach {$_.CapacityGB } | Measure-Object -sum).sum + $VMtomove.MemoryGB)
$opendatastores = $Datastores | where {($_.FreeSpaceGB-$VMSize/$_.CapacityGB)*100 -gt $PercentFree} | Sort FreeSpaceGB -Descending 
 
 
if (($opendatastores | Measure-Object).count -ge 1){
 
# Pause loop to wait on jobs to finish
while ((Get-Job -State Running).length -ge $MaxJobs){
Start-Sleep -seconds 3
}
 
#Determine Datastores in use
$DatastoresInUse = @()
$RunningJobs = Get-Job | where {$_.State -eq "Running"}
foreach ($OpenDatastore in $OpenDatastores){
foreach ($Job in $RunningJobs){
$JobArrayMatch = $JobArray | where {$_.JobID -eq $Job.ID} 
foreach ($Match in $JobArrayMatch | where {$_.Destination -eq $OpenDatastore}){
$DatastoresInUse += $OpenDatastore.name
}
}
}
 
$FilteredDatastores = @()
# Filter Datastore list based on those in use
foreach ($OpenDatastore in $OpenDatastores){
if ($DatastoresInUse -notcontains $OpenDatastore.Name){
$FilteredDatastores += $OpenDatastore
}
Else{}
}
 
# Determine Best Datastore detination 
foreach ($FilteredDatastore in $FilteredDatastores | Sort FreeSpaceMB -Descending){
if (((($FilteredDatastore.FreeSpaceGB-$VMSize)/$FilteredDatastore.CapacityGB)*100)-gt $PercentFree){
$DestinationDatastore = $FilteredDatastore
break
}
}
 
if ($DestinationDatastore -ne $null){
# Define Job Parameters
 
 
$jobSpec=@() 
$jobSpec += $jobcmd
$jobSpec += $vCenter 
$jobSpec += $saacount 
$jobSpec += $vi_pass 
$jobSpec += $VMtomove.ID
$jobSpec += $DestinationDatastore.Id
 
$StartJob = Start-Job -InputObject $jobSpec -ScriptBlock $jobSpec[0]
$Object = New-Object -TypeName psobject -Property @{
jobid = $StartJob.ID
destination = $DestinationDatastore
vm = $VM.Name
}
$JobArray += $object
Write-Host "Moving $VMtomove to $DestinationDatastore"
sleep 30
}
else{        
Write-Host "Cannot Move $VMtomove to a datastore accessable to the cluster, There is not enough space to maintain $PercentFree % criteria"            
}        
}
 
$i++
 
}
until(((((Get-Datastore $lowDS.Name).FreeSpaceGB+$VMdisksizeperDS)/(Get-Datastore $lowDS.Name).CapacityGB)*100) -gt 15)
do{
$i
$VMtomove = $VMslist["$i"]
$VMstoMove += $VMtomove
$VMdisksizeperDS = ($VMtomove |Get-HardDisk |?{$_.Filename -match $lowDS.Name}|foreach{$_.CapacityGB} |Measure-Object -Sum).sum
 
$Datastores = $VMHosts | Get-Datastore | where {($_.Name -notlike $DNIDSNames) -and ($_.Name -notlike $DNIDSNames1 ) -and ($_.Name -notlike $DNIDSNames2)} 
 
$VMSize = (($VMtomove |Get-HardDisk| Foreach {$_.CapacityGB } | Measure-Object -sum).sum + $VMtomove.MemoryGB)
$opendatastores = $Datastores | where {($_.FreeSpaceGB-$VMSize/$_.CapacityGB)*100 -gt $PercentFree} | Sort FreeSpaceGB -Descending 
 
 
if (($opendatastores | Measure-Object).count -ge 1){
 
# Pause loop to wait on jobs to finish
while ((Get-Job -State Running).length -ge $MaxJobs){
Start-Sleep -seconds 3
}
 
#Determine Datastores in use
$DatastoresInUse = @()
$RunningJobs = Get-Job | where {$_.State -eq "Running"}
foreach ($OpenDatastore in $OpenDatastores){
foreach ($Job in $RunningJobs){
$JobArrayMatch = $JobArray | where {$_.JobID -eq $Job.ID} 
foreach ($Match in $JobArrayMatch | where {$_.Destination -eq $OpenDatastore}){
$DatastoresInUse += $OpenDatastore.name
}
}
}
 
$FilteredDatastores = @()
# Filter Datastore list based on those in use
foreach ($OpenDatastore in $OpenDatastores){
if ($DatastoresInUse -notcontains $OpenDatastore.Name){
$FilteredDatastores += $OpenDatastore
}
Else{}
}
 
# Determine Best Datastore detination 
foreach ($FilteredDatastore in $FilteredDatastores | Sort FreeSpaceMB -Descending){
if (((($FilteredDatastore.FreeSpaceGB-$VMSize)/$FilteredDatastore.CapacityGB)*100)-gt $PercentFree){
$DestinationDatastore = $FilteredDatastore
break
}
}
 
if ($DestinationDatastore -ne $null){
# Define Job Parameters
 
 
$jobSpec=@() 
$jobSpec += $jobcmd
$jobSpec += $vCenter 
$jobSpec += $saacount 
$jobSpec += $vi_pass 
$jobSpec += $VMtomove.ID
$jobSpec += $DestinationDatastore.Id
 
$StartJob = Start-Job -InputObject $jobSpec -ScriptBlock $jobSpec[0]
$Object = New-Object -TypeName psobject -Property @{
jobid = $StartJob.ID
destination = $DestinationDatastore
vm = $VM.Name
}
$JobArray += $object
Write-Host "Moving $VMtomove to $DestinationDatastore"
sleep 30
}
else{        
Write-Host "Cannot Move $VMtomove to a datastore accessable to the cluster, There is not enough space to maintain $PercentFree % criteria"            
}        
}
 
$i++
 
}
until(((((Get-Datastore $lowDS.Name).FreeSpaceGB+$VMdisksizeperDS)/(Get-Datastore $lowDS.Name).CapacityGB)*100) -gt 15)
Regards, Satheesh

View solution in original post

0 Kudos
14 Replies
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

when i am trying to do the storage vmotion using below command it is working absolutely fine.

move-vm $vm -destination $vmhost -datastore $destdatastore,

i filtered the datastores with <15%, from each of those datastores, i am trying to do the storage vmotion to highest free space datastore, i am able to find the funciton for this.

but if i need to migrate multiple VMs at a time, then i am stuck with that place, and one by one only storage vmotion happening with the above commends, which is taking long time to get the data store balance.

Regards, Satheesh
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Try something like this

$dscName = 'MyDSC'

$dsc = Get-View -ViewType StoragePod -Filter @{'Name'=$dscName}

$si = Get-View ServiceInstance

$storMgr = Get-View -Id $si.Content.StorageResourceManager

$storMgr.RefreshStorageDrsRecommendation($dsc.MoRef)

$dsc.UpdateViewData()

if($dsc.PodStorageDrsEntry.Recommendation){

    $dsc.PodStorageDrsEntry.Recommendation | %{

        $storMgr.ApplyStorageDrsRecommendationToPod_Task($dsc.MoRef,$_.Key)

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

satheeshsatz
Enthusiast
Enthusiast
Jump to solution

hi LucD,

thank you for your response.

Actually, i was looking for storage vmotion by poewrcli script across the datastore in a cluster.

VMs should be balanced across the datastore and it should have 15% of free space on each datastore.

i have used move-vm with destination $datastore, but it has executing one by one, i would like to do more storage vmotion at a time(maximum 5).

Can you please help for this?"

Regards, Satheesh
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Perhaps I get you question not correctly, but balancing is exactly what SDRS is doing.

You configure the SDRS settings (space threshold and IO metric) tat govern this balancing.

The code I gave will force SDRS to look at the current situation, and eventually trigger some svMotions.

The number of svMotions is limited, see Limits on Simultaneous Migrations

It depends on your setup (# of datastores, # of hosts...) how many svMotions you can start in parallel.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

Sorry for not raising question understandable.

1. we are not suppose to enable SDRS, as our infra is huge with critical servers, which BU wont accept for storage vmotion by SDRS. So that, whenever we face datastore free space issue, we are perfroming storage vmotion from low free space datastore to high free space, this way we are balancing the storage free % on datastore.

2. For this, i have started working on the script to do this steps with the help of powercli, like as below.

     a. Will check the average datastore free space % on the cluster and if it is less than 20%, we will get new LUN for datastore.

     b. if the average datastore free space is more than 20%, then we will start do the svomotion from low free space DS to high freespace DS with powercli.

     c. for each datastore, it will check the free space and select a VM and perform svmotion, then it will check the source free % if it is more than 15%, then next low free space datastore will be considered for svmotion.

     D. Also, i will give input list of VMs which should not be migrated(critical VMs)

     e. if i do one bye with the same steps it will tkae long time, so i want to perform the same tasks for multiple svmotion(we have 10g hba cards and we can have max 5svmotion for us)

so i want to create powercli for the above conditions.

Regards, Satheesh
0 Kudos
LucD
Leadership
Leadership
Jump to solution

You could do something like the following.

But there might be some flaws in your logic:

  • while doing svMotion the Free Space % on the target datastore diminishes. You might end up in an eternal loop
  • Datastores which are the target of a running svMotions, don't reflect the actual free space correctly yet
  • The logic takes each of the datastores one by one, but the situation might change for a datastore that was already 'handled'

In any case, this could be something to start from

$clusterName = 'MyCluster'

$excludeVMFile = '.\exclude-VM.csv'

$maxParallelsvMotion = 5

$excludeVM = Import-Csv -Path $excludeVMFile -UseCulture | Select -ExpandProperty Name

$cluster = Get-Cluster -Name $clusterName

$datastores = Get-VMHost -Location $cluster | Get-Datastore |

    where{$_.ExtensionData.Summary.MultipleHostAccess} |

    Sort-Object -Property {$_.FreeSpaceGB/$_.CapacityGB}

$totalCapacity = $datastores | Measure-Object -Property CapacityGB -Sum | select -ExpandProperty Sum

$totalFree = $datastores | Measure-Object -Property FreeSpaceGB -Sum | select -ExpandProperty Sum

# Condition 2.a - Total free space less than 20%

if($totalFree/$totalCapacity -lt 0.2){

    Write-Host "Total free space less than 20%"

    Exit

}

# svMotion VMs if datastore has less than 15% free space

foreach($ds in $datastores){

    while(($ds.FreeSpaceGB/$ds.CapacityGB) -lt 0.15){

        $vm = Get-VM -Datastore $ds | where{$excludeVM -notcontains $_.Name} | Get-Random

        $dsTgt = $datastores |

            Sort-Object -Property {(Get-Datastore -Name $_.Name).FreeSpaceGB} -Descending |

            Select -First 1

        $countsvMotion = (Get-Task -Status Running,Queued | where{$_.Name -eq 'RelocateVM_Task'}).Count

        while($countsvMotion -ge $maxParallelsvMotion){

            sleep 5

        }

        Move-VM -VM $vm -Datastore $dsTgt -Confirm:$false -RunAsync

        $ds = Get-Datastore -Name $ds.Name

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

satheeshsatz
Enthusiast
Enthusiast
Jump to solution

Thank you for the rapid response and yes your right, my step has flaws and have pointed out those flaws.

my Flow going to be as below.

1. From Each datastore, i am collecting the VMs need to be moved to other datastore.(selecting the VMs from orginal freespace-vmdk size = new freespze, and it should continue until new free space is more than 15%), by this appending the vMs list

2. for each VMs move it to destination DS until it reach 15%

so instead of moving all the VMs at a time, i am moving it one by one by the strat-job command

Regards, Satheesh
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Sorry, not sure what you mean by 1).

How exactly do you pick the VMs that can be moved?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

Sorry Again for not making understandable question.

1. From each low space datastore, GET-VM |sort-object provisioningGB

     a. so that i will gave the VMs in list with sort of small size to max size.

     b. in that time i will append the $VMslist += $vm[0] then i will minus the size from total datastore(then if free space is >15% we will move on to next datastore, if not next VM will be append with the $VMlist. like that $VMslist will get increase until the free space is more than 15%)

2. Then for each $VM, will create job to initiate vmotion.

hope this makes sense.

Regards, Satheesh
0 Kudos
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

$VMstoMove = @()

foreach($lowDS in $ClosedDatastores) {

$VMslist = (Get-VM -Datastore $lowDS |?{$_.Name -notmatch $FilterText} |Sort-Object ProvisionedSpaceGB)

$i = 0

     do{

     $VMtomove = $VMslist[$i]

     $VMstoMove += $VMtomove

     $VMdisksizeperDS = (get-vm $VMtomove |Get-HardDisk |?{$_.Filename -match "$lowDS.Name"}|foreach{$_.CapacityGB} |Measure-Object -Sum).sum

     $i++

     }until(((((Get-Datastore $lowDS.Name).FreeSpaceGB+$VMdisksizeperDS)/(Get-Datastore $lowDS.Name).CapacityGB)*100,2) -gt "15")

}

Like the above i am getting the list of VMs to do svmotion has been creatd, however i am facing an error  as below.

_++++++++++++++++++++++++++++++++++++++++++++++++

Cannot convert argument "1", with value: "System.Object[]", for "op_Multiply" to

type "System.Decimal": "Cannot convert the "System.Object[]" value of type

"System.Object[]" to type "System.Decimal"."

At line:14 char:1

+ $i++

+ ~~~~

    + CategoryInfo          : NotSpecified: (:) [], MethodException

    + FullyQualifiedErrorId : MethodArgumentConversionInvalidCastArgument

Cannot convert argument "1", with value: "System.Object[]", for "op_Multiply" to

type "System.Decimal": "Cannot convert the "System.Object[]" value of type

"System.Object[]" to type "System.Decimal"."

At line:14 char:1

+ $i++

+ ~~~~

    + CategoryInfo          : NotSpecified: (:) [], MethodException

    + FullyQualifiedErrorId : MethodArgumentConversionInvalidCastArgument

_++++++++++++++++++++++++++++++++++++++++++++++++

after appending the list, iwill proceed wtih svmotion list

Regards, Satheesh
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Try like this

$VMstoMove = @()

foreach($lowDS in $ClosedDatastores){

    $VMslist = (Get-VM -Datastore $lowDS |?{$_.Name -notmatch $FilterText} |Sort-Object ProvisionedSpaceGB)

    

    $i = 0

    do{

        $ds = Get-Datastore -Name $lowDS.Name

        $VMtomove = $VMslist[$i]

        $VMstoMove += $VMtomove

        $VMdisksizeperDS = Get-VM $VMtomove |Get-HardDisk |

            Where-Object{$_.Filename -match "$lowDS.Name"}|

            Measure-Object -Property CapacityGB -Sum |

            select -ExpandProperty Sum

        $i++

    }until((($ds.FreeSpaceGB+$VMdisksizeperDS)/$ds.CapacityGB*100) -gt 15)

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

adamjg
Hot Shot
Hot Shot
Jump to solution

LucD did ask this question, but I think it needs to be asked again.  Are you saying that your org will not allow you to enable SDRS because it moves VMs without their knowledge, but they will allow you to run a script that does exactly the same thing?  If so, this should be an education issue and not something to solve with a script.

0 Kudos
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

hi thank you for your response, we too aware of how SDRS is working and capabilities. 

but still our management is not ready to accept at this point of time(even DRS is not enabled on some of the clusters(finance based VMs)).

Regards, Satheesh
0 Kudos
satheeshsatz
Enthusiast
Enthusiast
Jump to solution

Thanks a lot Brother, i was able to complete this with the below codes(ofcourse with your input too)

do{
$i
$VMtomove = $VMslist["$i"]
$VMstoMove += $VMtomove
$VMdisksizeperDS = ($VMtomove |Get-HardDisk |?{$_.Filename -match $lowDS.Name}|foreach{$_.CapacityGB} |Measure-Object -Sum).sum
 
$Datastores = $VMHosts | Get-Datastore | where {($_.Name -notlike $DNIDSNames) -and ($_.Name -notlike $DNIDSNames1 ) -and ($_.Name -notlike $DNIDSNames2)} 
 
$VMSize = (($VMtomove |Get-HardDisk| Foreach {$_.CapacityGB } | Measure-Object -sum).sum + $VMtomove.MemoryGB)
$opendatastores = $Datastores | where {($_.FreeSpaceGB-$VMSize/$_.CapacityGB)*100 -gt $PercentFree} | Sort FreeSpaceGB -Descending 
 
 
if (($opendatastores | Measure-Object).count -ge 1){
 
# Pause loop to wait on jobs to finish
while ((Get-Job -State Running).length -ge $MaxJobs){
Start-Sleep -seconds 3
}
 
#Determine Datastores in use
$DatastoresInUse = @()
$RunningJobs = Get-Job | where {$_.State -eq "Running"}
foreach ($OpenDatastore in $OpenDatastores){
foreach ($Job in $RunningJobs){
$JobArrayMatch = $JobArray | where {$_.JobID -eq $Job.ID} 
foreach ($Match in $JobArrayMatch | where {$_.Destination -eq $OpenDatastore}){
$DatastoresInUse += $OpenDatastore.name
}
}
}
 
$FilteredDatastores = @()
# Filter Datastore list based on those in use
foreach ($OpenDatastore in $OpenDatastores){
if ($DatastoresInUse -notcontains $OpenDatastore.Name){
$FilteredDatastores += $OpenDatastore
}
Else{}
}
 
# Determine Best Datastore detination 
foreach ($FilteredDatastore in $FilteredDatastores | Sort FreeSpaceMB -Descending){
if (((($FilteredDatastore.FreeSpaceGB-$VMSize)/$FilteredDatastore.CapacityGB)*100)-gt $PercentFree){
$DestinationDatastore = $FilteredDatastore
break
}
}
 
if ($DestinationDatastore -ne $null){
# Define Job Parameters
 
 
$jobSpec=@() 
$jobSpec += $jobcmd
$jobSpec += $vCenter 
$jobSpec += $saacount 
$jobSpec += $vi_pass 
$jobSpec += $VMtomove.ID
$jobSpec += $DestinationDatastore.Id
 
$StartJob = Start-Job -InputObject $jobSpec -ScriptBlock $jobSpec[0]
$Object = New-Object -TypeName psobject -Property @{
jobid = $StartJob.ID
destination = $DestinationDatastore
vm = $VM.Name
}
$JobArray += $object
Write-Host "Moving $VMtomove to $DestinationDatastore"
sleep 30
}
else{        
Write-Host "Cannot Move $VMtomove to a datastore accessable to the cluster, There is not enough space to maintain $PercentFree % criteria"            
}        
}
 
$i++
 
}
until(((((Get-Datastore $lowDS.Name).FreeSpaceGB+$VMdisksizeperDS)/(Get-Datastore $lowDS.Name).CapacityGB)*100) -gt 15)
do{
$i
$VMtomove = $VMslist["$i"]
$VMstoMove += $VMtomove
$VMdisksizeperDS = ($VMtomove |Get-HardDisk |?{$_.Filename -match $lowDS.Name}|foreach{$_.CapacityGB} |Measure-Object -Sum).sum
 
$Datastores = $VMHosts | Get-Datastore | where {($_.Name -notlike $DNIDSNames) -and ($_.Name -notlike $DNIDSNames1 ) -and ($_.Name -notlike $DNIDSNames2)} 
 
$VMSize = (($VMtomove |Get-HardDisk| Foreach {$_.CapacityGB } | Measure-Object -sum).sum + $VMtomove.MemoryGB)
$opendatastores = $Datastores | where {($_.FreeSpaceGB-$VMSize/$_.CapacityGB)*100 -gt $PercentFree} | Sort FreeSpaceGB -Descending 
 
 
if (($opendatastores | Measure-Object).count -ge 1){
 
# Pause loop to wait on jobs to finish
while ((Get-Job -State Running).length -ge $MaxJobs){
Start-Sleep -seconds 3
}
 
#Determine Datastores in use
$DatastoresInUse = @()
$RunningJobs = Get-Job | where {$_.State -eq "Running"}
foreach ($OpenDatastore in $OpenDatastores){
foreach ($Job in $RunningJobs){
$JobArrayMatch = $JobArray | where {$_.JobID -eq $Job.ID} 
foreach ($Match in $JobArrayMatch | where {$_.Destination -eq $OpenDatastore}){
$DatastoresInUse += $OpenDatastore.name
}
}
}
 
$FilteredDatastores = @()
# Filter Datastore list based on those in use
foreach ($OpenDatastore in $OpenDatastores){
if ($DatastoresInUse -notcontains $OpenDatastore.Name){
$FilteredDatastores += $OpenDatastore
}
Else{}
}
 
# Determine Best Datastore detination 
foreach ($FilteredDatastore in $FilteredDatastores | Sort FreeSpaceMB -Descending){
if (((($FilteredDatastore.FreeSpaceGB-$VMSize)/$FilteredDatastore.CapacityGB)*100)-gt $PercentFree){
$DestinationDatastore = $FilteredDatastore
break
}
}
 
if ($DestinationDatastore -ne $null){
# Define Job Parameters
 
 
$jobSpec=@() 
$jobSpec += $jobcmd
$jobSpec += $vCenter 
$jobSpec += $saacount 
$jobSpec += $vi_pass 
$jobSpec += $VMtomove.ID
$jobSpec += $DestinationDatastore.Id
 
$StartJob = Start-Job -InputObject $jobSpec -ScriptBlock $jobSpec[0]
$Object = New-Object -TypeName psobject -Property @{
jobid = $StartJob.ID
destination = $DestinationDatastore
vm = $VM.Name
}
$JobArray += $object
Write-Host "Moving $VMtomove to $DestinationDatastore"
sleep 30
}
else{        
Write-Host "Cannot Move $VMtomove to a datastore accessable to the cluster, There is not enough space to maintain $PercentFree % criteria"            
}        
}
 
$i++
 
}
until(((((Get-Datastore $lowDS.Name).FreeSpaceGB+$VMdisksizeperDS)/(Get-Datastore $lowDS.Name).CapacityGB)*100) -gt 15)
Regards, Satheesh
0 Kudos