Highlighted
Enthusiast
Enthusiast

PowerCLi script to Upload files into datastores for multiple hosts in parallel

Jump to solution

Hello Friends,

Recently i've got a scenario where i have to upload the files into multiple hosts in their local datastores around 2000+ boxes. I'm trying below scripts to upload the files to selected 50 hosts in parallel. but somehow script is not running as per our expectation. Any help would be highly appreciated here.

Script object: master script should initiate both file upload scripts in parallel for multiple hosts at a time.

Master script:

===========

$hosts= Get-Content "C:\HostList.txt"

foreach ($esx in $hosts) {

Write-Host "Running File upload1 script on host: $esx ..."  -ForegroundColor Yellow

#.\Test_FileUpload.ps1 -HostName $esx -User "XXXX"  -Pswd "XXXX"  

Write-Host "Running File upload2 script on host: $esx ..."  -ForegroundColor Yellow

#.\Test_FileUpload2.ps1 -HostName $esx -User "XXX"  -Pswd "XXXX"

}

script2:Test_FileUpload.ps1

===========

Param(

[string]$HostName,

[string]$User,

[string]$Pswd

)

$fileName = "C:\Users\New folder\DELL-T330\ESXi670-202008001.zip"

#$tgtFolder = 'ISO'

$StartTime = Get-Date

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Please wait while connecting the host: $HostName " -ForegroundColor Yellow

Write-Host "************************************************" -ForegroundColor Cyan

Connect-VIServer -Server $HostName -user $User -Password $Pswd

Write-Host "connected to the server: $HostName" -ForegroundColor Green

$datastores = Get-VMHost $HostName | Get-Datastore | where{$_.name -like "datastore*"}

    foreach($ds in $datastores){

   

    New-PSDrive -Location $ds -Name DS -PSProvider VimDatastore -Root "\" > $null

    Write-Host "New ISO folder has been created under DS: $ds " -ForegroundColor Green

     New-Item -Path DS:\ISO -ItemType Directory

    

     $tgtFolder ='ISO'

     New-PSDrive -Location $ds -Name DS1 -PSProvider VimDatastore -Root "\" > $null

  

    if(!(Test-Path -Path "DS1:/$($tgtFolder)")){

        New-Item -ItemType Directory -Path "DS1:/$($tgtFolder)" > $null

    }

    Write-Host "DS copy is in-progres.. Pls wait" -ForegroundColor Yellow

    Copy-DatastoreItem -Item $fileName -Destination "DS1:/$($tgtFolder)"

   

    Remove-PSDrive -Name DS -Confirm:$false

    Remove-PSDrive -Name DS1 -Confirm:$false

}

Write-Host " DS copy task has completed for host: $fileName" -ForegroundColor Green

$EndTime = Get-Date

$TotalTime = New-TimeSpan –Start $StartTime –End $EndTime #| select -expandproperty Seconds

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Total time taken to complete the ISO copy: $TotalTime " -ForegroundColor Green

Write-Host "************************************************" -ForegroundColor Cyan

script3:Test_FileUpload2

==========

Param(

[string]$HostName,

[string]$User,

[string]$Pswd

)

$fileName = "C:\Users\New folder\DELL-T330\VMware-VMvisor-Installer-6.7.0.update03-16316930.x86_64-DellEMC_Customized-A06.zip"

#$tgtFolder = 'ISO'

$StartTime = Get-Date

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Please wait while connecting the host: $HostName " -ForegroundColor Yellow

Write-Host "************************************************" -ForegroundColor Cyan

Connect-VIServer -Server $HostName -user $User -Password $Pswd

Write-Host "connected to the server: $HostName" -ForegroundColor Green

$datastores = Get-VMHost $HostName | Get-Datastore | where{$_.name -like "datastore*"}

    foreach($ds in $datastores){

   

    New-PSDrive -Location $ds -Name DS -PSProvider VimDatastore -Root "\" > $null

    Write-Host "New ISO folder has been created under DS: $ds " -ForegroundColor Green

     New-Item -Path DS:\ISO -ItemType Directory

    

     $tgtFolder ='ISO'

     New-PSDrive -Location $ds -Name DS1 -PSProvider VimDatastore -Root "\" > $null

  

    if(!(Test-Path -Path "DS1:/$($tgtFolder)")){

        New-Item -ItemType Directory -Path "DS1:/$($tgtFolder)" > $null

    }

    Write-Host "DS copy is in-progres.. Pls wait" -ForegroundColor Yellow

    Copy-DatastoreItem -Item $fileName -Destination "DS1:/$($tgtFolder)"

   

    Remove-PSDrive -Name DS -Confirm:$false

    Remove-PSDrive -Name DS1 -Confirm:$false

}

Write-Host " DS copy task has completed for host: $fileName" -ForegroundColor Green

$EndTime = Get-Date

$TotalTime = New-TimeSpan –Start $StartTime –End $EndTime #| select -expandproperty Seconds

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Total time taken to complete the ISO copy: $TotalTime " -ForegroundColor Green

Write-Host "************************************************" -ForegroundColor Cyan

Regards,

Swamy Naveen

1 Solution

Accepted Solutions
Highlighted
User Moderator
User Moderator

As it happens there was a recent thread on here that contains some skeleton code.
See Solved: Need help running start-job within loop... |VMware Communities


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
8 Replies
Highlighted
User Moderator
User Moderator

You could run multiple jobs in parallel in the background with Start-Job.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Highlighted
Enthusiast
Enthusiast

Hi LucD,

Thanks for the example. yeah, I've incorporated start-job in the script. it's working as expected. however, it doesn't showing jobs status. in order to check the status i had to run $jobs variable every time manually. Is there any option available to extract upload status into output file? also, After completion of file upload would like to have simple report across the hosts just to confirm that all the files are uploaded successfully.

Report should look like this:

==========

vCenter Name, Hostname, Datastorename ISO foldername, file name, Size of the file in GB

$User ="root"

$Pswd = "XXXX"

$hosts = Get-Content "C:\HostList.txt"

$Jobs = @()

$Output = @()

$sb1 = {

Param(

[string]$HostName,

[string]$User,

[string]$Pswd

)

$fileName = "C:\Users\New folder\DELL-T330\VMware-VMvisor-Installer-6.7.0.update03-16316930.x86_64-DellEMC_Customized-A06.zip"

#$tgtFolder = 'ISO'

$StartTime = Get-Date

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Please wait while connecting the host: $HostName " -ForegroundColor Yellow

Write-Host "************************************************" -ForegroundColor Cyan

Connect-VIServer -Server $HostName -user $User -Password $Pswd

Write-Host "connected to the server: $HostName" -ForegroundColor Green

$datastores = Get-VMHost $HostName | Get-Datastore | where{$_.name -like "datastore*"}

    foreach($ds in $datastores){

   

    New-PSDrive -Location $ds -Name DS -PSProvider VimDatastore -Root "\" > $null

    Write-Host "New ISO folder has been created under DS: $ds " -ForegroundColor Green

     New-Item -Path DS:\ISO -ItemType Directory

    

     $tgtFolder ='ISO'

     New-PSDrive -Location $ds -Name DS1 -PSProvider VimDatastore -Root "\" > $null

  

    if(!(Test-Path -Path "DS1:/$($tgtFolder)")){

        New-Item -ItemType Directory -Path "DS1:/$($tgtFolder)" > $null

    }

    Write-Host "DS copy is in-progres.. Pls wait" -ForegroundColor Yellow

    Copy-DatastoreItem -Item $fileName -Destination "DS1:/$($tgtFolder)"

   

    Remove-PSDrive -Name DS -Confirm:$false

    Remove-PSDrive -Name DS1 -Confirm:$false

}

}

foreach($HostName in $hosts)

{

    $Jobs += start-job -ScriptBlock $sb1 -ArgumentList $HostName,$User,$Pswd

}

$Jobs | Wait-Job

ForEach ($Job in $Jobs)

{   $Output += $Job | Receive-Job

    $Job | Remove-Job

}

$Output | Out-File "$($env:userprofile)\Desktop\output1.txt"

Write-Host " DS copy task has completed: $fileName" -ForegroundColor Green

$EndTime = Get-Date

$TotalTime = New-TimeSpan –Start $StartTime –End $EndTime #| select -expandproperty Seconds

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Total time taken to complete the file upload jobs: $TotalTime " -ForegroundColor Green

Write-Host "************************************************" -ForegroundColor Cyan

Regards,

Swamy Naveen

0 Kudos
Highlighted
User Moderator
User Moderator

I normally use the Receive-Job cmdlet with the Wait switch.

You can direct the object(s) returned or a set of properties on those objects to a file.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
Highlighted
Enthusiast
Enthusiast

Hi LucD,

Could you provide some examples with Receive-job cmdlet? As i am not sure where to add this cmdlet in my script.

Regards,

Naveen

0 Kudos
Highlighted
User Moderator
User Moderator

As it happens there was a recent thread on here that contains some skeleton code.
See Solved: Need help running start-job within loop... |VMware Communities


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
Highlighted
Enthusiast
Enthusiast

Thanks LucD for the help. currently i am using below script to validate the size of the ISO directory post completion of file upload. however, host name is missing. any suggestions would be really helpful.

Thank you in advance.

foreach ($esx in $hosts) {

Write-Host "************************************************" -ForegroundColor Cyan

Write-Host "Please wait while connecting the host: $esx " -ForegroundColor Yellow

Write-Host "************************************************" -ForegroundColor Cyan

Connect-VIServer -Server $esx -user "root" -Password "XXXX"

Write-Host "connected to the server: $esx" -ForegroundColor Green

$ds = Get-VMHost $esx  | Get-Datastore | where{$_.name -like "datastore*"}

New-PSDrive -Location $ds -Name DS -PSProvider VimDatastore -Root "\" > $null

  

    Get-ChildItem -Path DS: | where{$_.ItemType -eq 'Folder' -and $_.Name -like 'ISO'} | %{

        New-Object PSObject -Property @{

            hostname = $esx.name

            Datastore = $ds.Name

            Folder = $_.Name

            SizeGB = [math]::Round((Get-ChildItem -Path "DS:\$($_.Name)" | Measure-Object -Property Length -Sum | select -ExpandProperty Sum)/1MB,1)

        }

    }

    Remove-PSDrive -Name DS -Confirm:$false

    }

0 Kudos
Highlighted
User Moderator
User Moderator

You didn't say what is in $hosts, but I suspect it might be an array of ESXi nodenames.

In that case $esx holds a string and the expression $esx.Name returns $null.

Try using $esx instead of $esx.Name


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Highlighted
Enthusiast
Enthusiast

Sorry, I missed it. Yeah $hosts holds list of ESXi host name that we provide as input. thanks a lot for the update. I have changed it and it worked like a charm. I really appreciate your help on this.

    

DatastoreFolderhostnameSizeGB
datastore1ISOas1vm1378453.2
0 Kudos