1 2 Previous Next 22 Replies Latest reply on Aug 14, 2018 1:06 PM by tvo

    Datastore Clusters migration script

    tvo Novice

      Hello,

       

      I need some help with a few PowerCLI scripts for my vCenter migration project specifically with datastores.  I would like once script that will export all the folders, datastore clusters and it's members to either csv or xml for a specified virtual datacenter.  Then another script to import the exported csv/xml to a specified virtual datacenter.  I will run the export script on my old vCenter then run the import script on my new vCenter after I've migrated the ESXi hosts over.  Attached is s sample screen shot of the folders and datastore clusters/members.  Thanks in advance for any help.

       

      datastores.png

        • 1. Re: Datastore Clusters migration script
          LucD Guru
          User ModeratorsvExpertCommunity Warriors

          There are quite a few posts in the wild that provide scripts for such a migration.

          Have for example a look at Gabrie's Cheap disaster recovery

          Let us know which parts you are still missing.

          • 2. Re: Datastore Clusters migration script
            tvo Novice

            Thanks LucD, I've looked at the scripts in CheapDiasterRecovery and don't see anything in regards to datastore clusters and datastore migrations unless I missed it.

            • 3. Re: Datastore Clusters migration script
              LucD Guru
              Community WarriorsvExpertUser Moderators

              The following 2 scripts will do an export/import of datastoreclusters.

              The import script assumes that the datastores exist.

              The scripts do not handle SDRS rules.

               

              The export

               

              Get-DatastoreCluster |

              Select -Property Name,SpaceUtilizationThresholdPercent,

                  IOLatencyThresholdMillisecond,

                  IOLoadBalanceEnabled,

                  SdrsAutomationLevel,

                  @{N='DSCPath';E={

                      $path = $_.Name

                      $parent = $_.ExtensionData.Parent

                      while($parent){

                          $parentObj = Get-View -Id $parent -Property Name,Parent

                          if('Datacenters','datastore' -notcontains $parentObj.Name){

                              $path = "$($parentObj.Name)/$path"

                          }

                          $parent = $parentObj.Parent

                      }

                      $path

                  }},

                  @{N='Datastores';E={(Get-Datastore -Location $_).Name -join '|'}} |

              Export-Csv -Path .\dsc-info.csv -NoTypeInformation -UseCulture

               

              The import

               

              Import-Csv -Path .\dsc-info.csv -UseCulture |

              ForEach-Object -Process {

                  $q = $_.DScPath.Split('/')

                  $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

                  $q[1..($q.Count - 2)] | %{

                      Try{

                          $folder = Get-Folder -Name $_ -Location $parent -Type Datastore -ErrorAction Stop

                      }

                      catch{

                          $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

                      }

                  }

                  try{

                      $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder

                  }

                  Catch{

                      $dsc = New-DatastoreCluster -Name $_.Name -Location $folder -Confirm:$false

                  }

               

                  $sDSC = @{

                      DatastoreCluster = $dsc

                      IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

                      IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

                      SdrsAutomationLevel = $_.SdrsAutomationLevel

                      SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

                      WhatIf = $true

                  }

                  Set-DatastoreCluster @sDSC

                  $_.Datastores.Split('|') | ForEach-Object -Process {

                      Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

                  }

              }

              • 4. Re: Datastore Clusters migration script
                tvo Novice

                Thanks LucD for the scripts!  Is there a way for me to specify the datacenter name I want to export/import?  I have multiple datacenters in my vCenter.

                • 5. Re: Datastore Clusters migration script
                  LucD Guru
                  Community WarriorsUser ModeratorsvExpert

                  You can change the 1st line of the export script to

                   

                  Get-Datacenter -Name MyDC | Get-DatastoreCluster |

                   

                  • 6. Re: Datastore Clusters migration script
                    tvo Novice

                    The export script works great now!  I have an issue with the import script though.  When I run the import script it errors out with the following.  Looks like it's looking for the original folders and datastore cluster in the exported csv instead of creating them on the new vCenter.  If I pre-create the folder and datastore cluster with the same names on the new vCenter the import script works and moves the datastore to the datastore cluster.

                     

                    Get-DatastoreCluster : 8/7/2018 4:26:54 PM      Get-DatastoreCluster            DatastoreCluster with name 'VCSE' was not found using the specified filter(s).

                    At C:\temp7\Datastore_Import.ps1:27 char:16

                    +         $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder

                    +                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                        + CategoryInfo          : ObjectNotFound: (:) [Get-DatastoreCluster], VimException

                        + FullyQualifiedErrorId : Core_OutputHelper_WriteNotFoundError,VMware.VimAutomation.ViCore.Cmdlets.Commands.GetDatastoreCluster

                     

                     

                    Set-DatastoreCluster : Cannot bind parameter 'DatastoreCluster'. Cannot convert the "" value of type "System.Management.Automation.PSCustomObject" to type

                    "VMware.VimAutomation.ViCore.Types.V1.DatastoreManagement.DatastoreCluster".

                    At C:\temp7\Datastore_Import.ps1:55 char:26

                    +     Set-DatastoreCluster @sDSC

                    +                          ~~~~~

                        + CategoryInfo          : InvalidArgument: (:) [Set-DatastoreCluster], ParameterBindingException

                        + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

                     

                     

                    Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

                    At C:\temp7\Datastore_Import.ps1:59 char:51

                    +         Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$fals ...

                    +                                                   ~~~~

                        + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

                        + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

                    • 7. Re: Datastore Clusters migration script
                      LucD Guru
                      Community WarriorsvExpertUser Moderators

                      Seems I forgot the ErrorAction on the Get-DatastoreCluster.

                      Replace that line with

                       

                         $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder -ErrorAction Stop

                       

                      • 8. Re: Datastore Clusters migration script
                        tvo Novice

                        Modified the  $dsc = Get-DatastoreCluster -Name $_.Name -Location $folder -ErrorAction Stop and ran the import script.  Got the following error.

                         

                        New-Folder : 8/8/2018 10:44:24 AM       New-Folder              '8/8/2018 10:44:24 AM   Get-Folder              Folder with name 'sjc_vcse_user_rw' was not ...' is invalid or exceeds the maximum number of

                        characters permitted.

                        At C:\temp7\Datastore_Import.ps1:19 char:23

                        + ...       $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

                        +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                            + CategoryInfo          : NotSpecified: (:) [New-Folder], InvalidName

                            + FullyQualifiedErrorId : Client20_InventoryServiceImpl_NewFolder_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewFolder

                         

                         

                        New-DatastoreCluster : Cannot validate argument on parameter 'Name'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

                        At C:\temp7\Datastore_Import.ps1:33 char:43

                        +         $dsc = New-DatastoreCluster -Name $_.Name -Location $folder - ...

                        +                                           ~~~~~~~

                            + CategoryInfo          : InvalidData: (:) [New-DatastoreCluster], ParameterBindingValidationException

                            + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewDatastoreCluster

                         

                         

                        Set-DatastoreCluster : 8/8/2018 10:44:24 AM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

                        At C:\temp7\Datastore_Import.ps1:55 char:5

                        +     Set-DatastoreCluster @sDSC

                        +     ~~~~~~~~~~~~~~~~~~~~~~~~~~

                            + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

                            + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

                         

                         

                        Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

                        At C:\temp7\Datastore_Import.ps1:59 char:51

                        +         Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$fals ...

                        +                                                   ~~~~

                            + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

                            + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

                        • 9. Re: Datastore Clusters migration script
                          LucD Guru
                          User ModeratorsvExpertCommunity Warriors

                          Seems my logic in the import script was not correct.

                          Try this revised version

                           

                          Import-Csv -Path .\dsc-info.csv -UseCulture |

                          ForEach-Object -Process {

                             $q = $_.DScPath.Split('/')

                             $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

                             $nrFolders = $q.Count - 2

                             if($nrFolders -gt 0){

                             $q[1..$nrFolders] | %{

                             Try{

                             $folder = Get-Folder -Name $_ -Location $parent -Type Datastore -ErrorAction Stop

                             }

                             catch{

                             $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

                             }

                             }

                             $parent = $folder

                             }

                             try{

                             $dsc = Get-DatastoreCluster -Name $q[-1] -Location $parent -ErrorAction Stop

                             }

                             Catch{

                             $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confirm:$false

                             }

                             $sDSC = @{

                             DatastoreCluster = $dsc

                             IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

                             IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

                             SdrsAutomationLevel = $_.SdrsAutomationLevel

                             SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

                             WhatIf = $true

                             }

                             Set-DatastoreCluster @sDSC

                             if($_.Datastores){

                             $_.Datastores.Split('|') | ForEach-Object -Process {

                             Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

                             }

                             }

                          }

                          • 10. Re: Datastore Clusters migration script
                            tvo Novice

                            Tried the new import script and got the following error:

                             

                            New-Folder : 8/8/2018 12:26:52 PM       New-Folder              '8/8/2018 12:26:52 PM   Get-Folder              Folder with name 'sjc_vcse_user_rw' was not ...' is invalid or exceeds the maximum number of characters permitted.

                            At C:\temp7\Datastore_Import_2.ps1:22 char:14

                            +    $folder = New-Folder -Name $_ -Location $parent -Confirm:$false

                            +              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                                + CategoryInfo          : NotSpecified: (:) [New-Folder], InvalidName

                                + FullyQualifiedErrorId : Client20_InventoryServiceImpl_NewFolder_ViError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewFolder

                             

                             

                            New-DatastoreCluster : 8/8/2018 12:26:53 PM     New-DatastoreCluster            Value cannot be found for the mandatory parameter Location

                            At C:\temp7\Datastore_Import_2.ps1:40 char:11

                            +    $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confir ...

                            +           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                                + CategoryInfo          : NotSpecified: (:) [New-DatastoreCluster], VimException

                                + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.NewDatastoreCluster

                             

                             

                            Set-DatastoreCluster : 8/8/2018 12:26:53 PM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

                            At C:\temp7\Datastore_Import_2.ps1:60 char:4

                            +    Set-DatastoreCluster @sDSC

                            +    ~~~~~~~~~~~~~~~~~~~~~~~~~~

                                + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

                                + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

                             

                             

                            Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

                            At C:\temp7\Datastore_Import_2.ps1:66 char:46

                            +    Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

                            +                                              ~~~~

                                + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

                                + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

                            • 11. Re: Datastore Clusters migration script
                              LucD Guru
                              Community WarriorsUser ModeratorsvExpert

                              Again the 1st error is the one to look at, the others are consequences of the 1st one.
                              This folder ''sjc_vcse_user_rw''

                              • does it already exist?
                              • where in the path to the DSC is it located? The line(s) of the export CSV with this folder could clarify

                               

                              Or perhaps a screenshot where I can see where this folder is located on the source environment?

                              • 12. Re: Datastore Clusters migration script
                                tvo Novice

                                The folder "sjc_vcse_user_rw" does not exist in the new vCenter.  The DSC is located in the folder "sjc_vcse_user_rw".

                                 

                                Here's the DSCPath: AMER Sales & Training/sjc_vcse_user_rw/VCSE

                                 

                                Screenshot of what the source vCenter looks like:

                                 

                                datastores2.png

                                • 13. Re: Datastore Clusters migration script
                                  LucD Guru
                                  vExpertCommunity WarriorsUser Moderators

                                  Think I found the issue, I was using the ForEach pipeline variable ($_) in a Catch block.
                                  Which doesn't work of course.

                                  Try like this

                                   

                                  Import-Csv -Path .\dsc-info.csv -UseCulture |

                                  ForEach-Object -Process {

                                     $q = $DScPath.Split('/')

                                     $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Name $q[0])

                                     $nrFolders = $q.Count - 2

                                     if($nrFolders -gt 0){

                                     foreach($folderName in $q[1..$nrFolders]){

                                     Try{

                                     $folder = Get-Folder -Name $folderName -Location $parent -Type Datastore -ErrorAction Stop

                                     }

                                     catch{

                                     $folder = New-Folder -Name $folderName -Location $parent -Confirm:$false

                                     }

                                     }

                                     $parent = $folder

                                     }

                                     try{

                                     $dsc = Get-DatastoreCluster -Name $q[-1] -Location $parent -ErrorAction Stop

                                     }

                                     Catch{

                                     $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confirm:$false

                                     }

                                     $sDSC = @{

                                     DatastoreCluster = $dsc

                                     IOLatencyThresholdMillisecond = $_.IOLatencyThresholdMillisecond

                                     IOLoadBalanceEnabled = [Boolean]$_.IOLoadBalanceEnabled

                                     SdrsAutomationLevel = $_.SdrsAutomationLevel

                                     SpaceUtilizationThresholdPercent = $_.SpaceUtilizationThresholdPercent

                                     WhatIf = $true

                                     }

                                     Set-DatastoreCluster @sDSC

                                     if($_.Datastores){

                                     $_.Datastores.Split('|') | ForEach-Object -Process {

                                     Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

                                     }

                                     }

                                  }

                                  • 14. Re: Datastore Clusters migration script
                                    tvo Novice

                                    Ran the latest import script and it gave the following error:

                                     

                                    You cannot call a method on a null-valued expression.

                                    At C:\temp7\Datastore_Import_3.ps1:4 char:4

                                    +    $q = $DScPath.Split('/')

                                    +    ~~~~~~~~~~~~~~~~~~~~~~~~

                                        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

                                        + FullyQualifiedErrorId : InvokeMethodOnNull

                                     

                                     

                                    Cannot index into a null array.

                                    At C:\temp7\Datastore_Import_3.ps1:6 char:4

                                    +    $parent = Get-Folder -Name datastore -Location (Get-Datacenter -Na ...

                                    +    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                                        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

                                        + FullyQualifiedErrorId : NullArray

                                     

                                     

                                    Cannot index into a null array.

                                    At C:\temp7\Datastore_Import_3.ps1:40 char:4

                                    +    $dsc = New-DatastoreCluster -Name $q[-1] -Location $parent -Confir ...

                                    +    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                                        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException

                                        + FullyQualifiedErrorId : NullArray

                                     

                                     

                                    Set-DatastoreCluster : 8/9/2018 12:47:27 PM     Set-DatastoreCluster            Value cannot be found for the mandatory parameter DatastoreCluster

                                    At C:\temp7\Datastore_Import_3.ps1:60 char:4

                                    +    Set-DatastoreCluster @sDSC

                                    +    ~~~~~~~~~~~~~~~~~~~~~~~~~~

                                        + CategoryInfo          : NotSpecified: (:) [Set-DatastoreCluster], VimException

                                        + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomation.ViCore.Cmdlets.Commands.SetDatastoreCluster

                                     

                                     

                                    Move-Datastore : Cannot validate argument on parameter 'Destination'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again.

                                    At C:\temp7\Datastore_Import_3.ps1:66 char:46

                                    +    Move-Datastore -Datastore $_ -Destination $dsc -Confirm:$false

                                    +                                              ~~~~

                                        + CategoryInfo          : InvalidData: (:) [Move-Datastore], ParameterBindingValidationException

                                        + FullyQualifiedErrorId : ParameterArgumentValidationError,VMware.VimAutomation.ViCore.Cmdlets.Commands.MoveDatastore

                                    1 2 Previous Next