VMware Cloud Community
tvo
Contributor
Contributor

Datastore Clusters migration script

Hello,

I need some help with a few PowerCLI scripts for my vCenter migration project specifically with datastores.  I would like once script that will export all the folders, datastore clusters and it's members to either csv or xml for a specified virtual datacenter.  Then another script to import the exported csv/xml to a specified virtual datacenter.  I will run the export script on my old vCenter then run the import script on my new vCenter after I've migrated the ESXi hosts over.  Attached is s sample screen shot of the folders and datastore clusters/members.  Thanks in advance for any help.

datastores.png

Reply
0 Kudos
22 Replies
LucD
Leadership
Leadership

There are quite a few posts in the wild that provide scripts for such a migration.

Have for example a look at Gabrie's Cheap disaster recovery

Let us know which parts you are still missing.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Thanks LucD, I've looked at the scripts in CheapDiasterRecovery and don't see anything in regards to datastore clusters and datastore migrations unless I missed it.

Reply
0 Kudos
LucD
Leadership
Leadership

The following 2 scripts will do an export/import of datastoreclusters.

The import script assumes that the datastores exist.

The scripts do not handle SDRS rules.

The export

Get-DatastoreCluster |

Select -Property Name,SpaceUtilizationThresholdPercent,

    IOLatencyThresholdMillisecond,

    IOLoadBalanceEnabled,

    SdrsAutomationLevel,

    @{N='DSCPath';E={

        $path = $_.Name

        $parent = $_.ExtensionData.Parent

        while($parent){

            $parentObj = Get-View -Id $parent -Property Name,Parent

            if('Datacenters','datastore' -notcontains $parentObj.Name){

                $path = "$($parentObj.Name)/$path"

            }

            $parent = $parentObj.Parent

        }

        $path

    }},

    @{N='Datastores';E={(Get-Datastore -Location $_).Name -join '|'}} |

Export-Csv -Path .\dsc-info.csv -NoTypeInformation -UseCulture

The import

Import-Csv -Path .\dsc-info.csv -UseCulture |

ForEach-Object -Process {

    $q = $_.DScPath.Split('/')

    $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

    $q[1..($q.Count - 2)] | %{

        Try{

            $folder = Get-Folder -Name $_ -Location $parent -Type Datastore -ErrorAction Stop

        }

        catch{

            $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

        }

    }

    try{

        $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder

    }

    Catch{

        $dsc = New-DatastoreCluster -Name $_.Name -Location $folder -Confirm:$false

    }

    $sDSC = @{

        DatastoreCluster = $dsc

        IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

        IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

        SdrsAutomationLevel = $_.SdrsAutomationLevel

        SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

        WhatIf = $true

    }

    Set-DatastoreCluster @sDSC

    $_.Datastores.Split('|') | ForEach-Object -Process {

        Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

    }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Thanks LucD for the scripts!  Is there a way for me to specify the datacenter name I want to export/import?  I have multiple datacenters in my vCenter.

Reply
0 Kudos
LucD
Leadership
Leadership

You can change the 1st line of the export script to

Get-Datacenter -Name MyDC | Get-DatastoreCluster |


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

The export script works great now!  I have an issue with the import script though.  When I run the import script it errors out with the following.  Looks like it's looking for the original folders and datastore cluster in the exported csv instead of creating them on the new vCenter.  If I pre-create the folder and datastore cluster with the same names on the new vCenter the import script works and moves the datastore to the datastore cluster.

Get-DatastoreCluster : 8/7/2018 4:26:54 PM      Get-DatastoreCluster            DatastoreCluster with name 'VCSE' was not found using the specified filter(s).

At C:\temp7\Datastore_Import.ps1:27 char:16

+         $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder

+                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : ObjectNotFound: (:) [Get-DatastoreCluster], VimException

    + FullyQualifiedErrorId : Core_OutputHelper_WriteNotFoundError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastoreCluster

Set-DatastoreCluster : Cannot bind parameter 'DatastoreCluster'. Cannot convert the "" value of type "System.Management.Automation.PSCustomObject" to type

"VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.DatastoreCluster".

At C:\temp7\Datastore_Import.ps1:55 char:26

+     Set-DatastoreCluster @sDSC

+                          ~~~~~

    + CategoryInfo          : InvalidArgument: (:) [Set-DatastoreCluster], ParameterBindingException

    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

At C:\temp7\Datastore_Import.ps1:59 char:51

+         Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$fals ...

+                                                   ~~~~

    + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

Reply
0 Kudos
LucD
Leadership
Leadership

Seems I forgot the ErrorAction on the Get-DatastoreCluster.

Replace that line with

   $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder -ErrorAction Stop


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Modified the  $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder -ErrorAction Stop and ran the import script.  Got the following error.

New-Folder : 8/8/2018 10:44:24 AM       New-Folder              '8/8/2018 10:44:24 AM   Get-Folder              Folder with name 'sjc_vcse_user_rw' was not ...' is invalid or exceeds the maximum number of

characters permitted.

At C:\temp7\Datastore_Import.ps1:19 char:23

+ ...       $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

+                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [New-Folder], InvalidName

    + FullyQualifiedErrorId : Client20_InventoryServiceImpl_NewFolder_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewFolder

New-DatastoreCluster : Cannot validate argument on parameter 'Name'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

At C:\temp7\Datastore_Import.ps1:33 char:43

+         $dsc = New-DatastoreCluster -Name $_.Name -Location $folder - ...

+                                           ~~~~~~~

    + CategoryInfo          : InvalidData: (:) [New-DatastoreCluster], ParameterBindingValidationException

    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewDatastoreCluster

Set-DatastoreCluster : 8/8/2018 10:44:24 AM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

At C:\temp7\Datastore_Import.ps1:55 char:5

+     Set-DatastoreCluster @sDSC

+     ~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

At C:\temp7\Datastore_Import.ps1:59 char:51

+         Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$fals ...

+                                                   ~~~~

    + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

Reply
0 Kudos
LucD
Leadership
Leadership

Seems my logic in the import script was not correct.

Try this revised version

Import-Csv -Path .\dsc-info.csv -UseCulture |

ForEach-Object -Process {

   $q = $_.DScPath.Split('/')

   $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

   $nrFolders = $q.Count - 2

   if($nrFolders -gt 0){

   $q[1..$nrFolders] | %{

   Try{

   $folder = Get-Folder -Name $_ -Location $parent -Type Datastore -ErrorAction Stop

   }

   catch{

   $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

   }

   }

   $parent = $folder

   }

   try{

   $dsc = Get-DatastoreCluster -Name $q[-1] -Location $parent -ErrorAction Stop

   }

   Catch{

   $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confirm:$false

   }

   $sDSC = @{

   DatastoreCluster = $dsc

   IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

   IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

   SdrsAutomationLevel = $_.SdrsAutomationLevel

   SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

   WhatIf = $true

   }

   Set-DatastoreCluster @sDSC

   if($_.Datastores){

   $_.Datastores.Split('|') | ForEach-Object -Process {

   Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

   }

   }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Tried the new import script and got the following error:

New-Folder : 8/8/2018 12:26:52 PM       New-Folder              '8/8/2018 12:26:52 PM   Get-Folder              Folder with name 'sjc_vcse_user_rw' was not ...' is invalid or exceeds the maximum number of characters permitted.

At C:\temp7\Datastore_Import_2.ps1:22 char:14

+    $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

+              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [New-Folder], InvalidName

    + FullyQualifiedErrorId : Client20_InventoryServiceImpl_NewFolder_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewFolder

New-DatastoreCluster : 8/8/2018 12:26:53 PM     New-DatastoreCluster            Value cannot be found for the mandatory parameter Location

At C:\temp7\Datastore_Import_2.ps1:40 char:11

+    $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confir ...

+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [New-DatastoreCluster], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewDatastoreCluster

Set-DatastoreCluster : 8/8/2018 12:26:53 PM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

At C:\temp7\Datastore_Import_2.ps1:60 char:4

+    Set-DatastoreCluster @sDSC

+    ~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

At C:\temp7\Datastore_Import_2.ps1:66 char:46

+    Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

+                                              ~~~~

    + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

Reply
0 Kudos
LucD
Leadership
Leadership

Again the 1st error is the one to look at, the others are consequences of the 1st one.
This folder ''sjc_vcse_user_rw''

  • does it already exist?
  • where in the path to the DSC is it located? The line(s) of the export CSV with this folder could clarify

Or perhaps a screenshot where I can see where this folder is located on the source environment?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

The folder "sjc_vcse_user_rw" does not exist in the new vCenter.  The DSC is located in the folder "sjc_vcse_user_rw".

Here's the DSCPath: AMER Sales & Training/sjc_vcse_user_rw/VCSE

Screenshot of what the source vCenter looks like:

datastores2.png

Reply
0 Kudos
LucD
Leadership
Leadership

Think I found the issue, I was using the ForEach pipeline variable ($_) in a Catch block.
Which doesn't work of course.

Try like this

Import-Csv -Path .\dsc-info.csv -UseCulture |

ForEach-Object -Process {

   $q = $DScPath.Split('/')

   $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

   $nrFolders = $q.Count - 2

   if($nrFolders -gt 0){

   foreach($folderName in $q[1..$nrFolders]){

   Try{

   $folder = Get-Folder -Name $folderName -Location $parent -Type Datastore -ErrorAction Stop

   }

   catch{

   $folder = New-Folder -Name $folderName -Location $parent -Confirm:$false

   }

   }

   $parent = $folder

   }

   try{

   $dsc = Get-DatastoreCluster -Name $q[-1] -Location $parent -ErrorAction Stop

   }

   Catch{

   $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confirm:$false

   }

   $sDSC = @{

   DatastoreCluster = $dsc

   IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

   IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

   SdrsAutomationLevel = $_.SdrsAutomationLevel

   SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

   WhatIf = $true

   }

   Set-DatastoreCluster @sDSC

   if($_.Datastores){

   $_.Datastores.Split('|') | ForEach-Object -Process {

   Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

   }

   }

}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Ran the latest import script and it gave the following error:

You cannot call a method on a null-valued expression.

At C:\temp7\Datastore_Import_3.ps1:4 char:4

+    $q = $DScPath.Split('/')

+    ~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

    + FullyQualifiedErrorId : InvokeMethodOnNull

Cannot index into a null array.

At C:\temp7\Datastore_Import_3.ps1:6 char:4

+    $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Na ...

+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

    + FullyQualifiedErrorId : NullArray

Cannot index into a null array.

At C:\temp7\Datastore_Import_3.ps1:40 char:4

+    $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confir ...

+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

    + FullyQualifiedErrorId : NullArray

Set-DatastoreCluster : 8/9/2018 12:47:27 PM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

At C:\temp7\Datastore_Import_3.ps1:60 char:4

+    Set-DatastoreCluster @sDSC

+    ~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

At C:\temp7\Datastore_Import_3.ps1:66 char:46

+    Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

+                                              ~~~~

    + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

    + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

Reply
0 Kudos
LucD
Leadership
Leadership

Do you have rows in the CSV where the DSCPath column is empty?

Or perhaps empty rows in the CSV?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

No empty columns or rows.   Attached is the cvs file.

Reply
0 Kudos
LucD
Leadership
Leadership

Just tried with your CSV file, and the latest import script seems to work perfectly for me.

Are you still getting the error that DSCPath is $null?


Just to make sure, the Datacenter "AMER Sales & Training" already exists on the target vCenter?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
tvo
Contributor
Contributor

Yes, the "AMER Sales & Training" already exists on the target vCenter.  Thanks for testing and all your help LucD!  I'll build a brand new vCenter and will test.

Reply
0 Kudos
LucD
Leadership
Leadership

Just to verify that the import of the CSV is working correctly, can you do

Import-Csv -Path .\dsc-info.csv -UseCulture

and show me the output?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos