VMware Cloud Community
Gaurav_Awasthi
Enthusiast
Enthusiast

Parallel run of scripts for multiple vCenter and common output file

Hi,

I have a script which is executed on multiple vCenter Servers and exports the cluster capacity results to a CSV file.However since it executes on multiple vCenters sequentially the execution time is high.Is there a way it can be executed in parallel and then output be saved to one file.

Reply
0 Kudos
8 Replies
LucD
Leadership
Leadership

You can use background jobs with Start-Job.

See my Running a background job post for details..


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
Gaurav_Awasthi
Enthusiast
Enthusiast

Thanks Luc ,let my try it

Reply
0 Kudos
Gaurav_Awasthi
Enthusiast
Enthusiast

Hi Luc ,

I tried using background jobs and it works as expected.However I need to get the output of the commands exported to one csv file when pulling data from multiple connected vCenter servers.How do I achieve that ? I tried with below code but csv  file does not have the required data.

Is it  probably due to the type of data returned by Receive-job cmdlet?

Gettype() shows return type as

IsPublic IsSerial Name                    BaseType

True     True     Object[]                    System.Array

while get-member displays type as

TypeName: Deserialized.VMware.VimAutomation.ViCore.Impl.V1.VIServerImpl

which otherwise is TypeName: VMware.VimAutomation.ViCore.Impl.V1.VIServerImpl for Get-VM cmdlet

=======================================================================================

$code = {

param(

[string]$Server,

[string]$SessionId

)

Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -Confirm:$false | Out-Null

Connect-VIServer -Server $Server -Session $SessionId

Get-VM |Select Name,Powerstate,@{N="IP";E={$_.extensiondata.Guest.IPaddress}},@{N="FQDN";E={$_.extensiondata.Guest.Hostname}},@{N="OS";E={$_.ExtensionData.Guest.GuestFullName}},@{N="vCenter Service URL";E={$_.extensiondata.client.serviceurl}},@{N="Cluster";E={(get-vm $_|get-cluster).Name}},@{N="ESXi";E={(get-vm $_ |Get-VMHost).Name}}

}

$sJOb1 = @{

ScriptBlock = $code

ArgumentList = $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId

}

$sJOb2 = @{

ScriptBlock = $code

ArgumentList = $global:DefaultVIServers.Name[1], $global:DefaultVIServers.SessionId[1]

}

$j1=Start-Job @sJob1

$j2=start-job @sjob2

Receive-Job -Job $j1 -Wait |export-csv "C:\temp\test.csv" -Append

Receive-Job -Job $j2 -Wait |export-csv "c:\temp\test.csv" -Append

Reply
0 Kudos
LucD
Leadership
Leadership

When I run multiple background jobs, I normally use code like the following.

The Receive-Job for all launched background jobs seems to do the trick for me.

$code = {

    param(

        [string]$Server,

        [string]$SessionId,

        [PSObject[]]$VMObj

    )


    Start-Sleep -Milliseconds (Get-Random -Minimum 0 -Maximum 1000)

    Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -Confirm:$false | Out-Null


    Connect-VIServer -Server $Server -Session $SessionId | Out-Null


    $vmObj | ForEach-Object -Process {

        New-Object -TypeName Psobject -Property ([ordered]@{

            Name = $_.Name

            NumCpu = $_.NumCpu

            MemoryGB = $_.MemoryGB

        })

    }

}


$nrJobs = 4

$vms = Get-VM

$sliceSize = [math]::Ceiling($vms.Count/$nrJobs)


$sJOb = @{

    ScriptBlock = $code

    ArgumentList = $global:DefaultVIServer.Name, $global:DefaultVIServer.SessionId, $vmSlice, $report

}


$jobs = @()

1..$nrJobs| ForEach-Object -Process {

    "From: $(($_-1)*$sliceSize)  To: $([Math]::Min(($_*$sliceSize)-1,$vms.Count))"

    $sJob.ArgumentList[2] = $vms[(($_-1)*$sliceSize)..([Math]::Min(($_*$sliceSize)-1,$vms.Count))]

    $jobs += Start-Job @sJob

}


Receive-Job -Job $jobs -Wait -AutoRemoveJob |

Select -Property Name,NumCpu,MemoryGB |

Export-Csv -Path .\report.csv -UseCulture -NoTypeInformation


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
swarajbrp
Contributor
Contributor

Hello LucD,

I need your help in below script to reduce the output time. we have almost 50 VCs in our environment and the output is taking much much time. Can you please help.

 

Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')
Connect-VIServer -Server ****************** -User administrator@vsphere.local -Password ('******************')

 


$logFile = "******************.CSV" # Where you want to save the CSV file
$vcsCluster = "*" # Filters which cluster you want to pull VM stats from.


Import-Module VMware.VimAutomation.Core

Set-PowerCLIConfiguration -InvalidCertificateAction Ignore -Confirm:$false
set-powercliconfiguration -scope user -confirm:$false

$vms = Get-Cluster -Name $vcsCluster | Get-VM
$count = 0
$results = @()
$Script:ProgressPreferenceOriginal = $ProgressPreference
Clear-Host
foreach($vm in $vms){
# Progress Bar setup
$count++
$percentComplete = [math]::Round(($count / $vms.Count) * 100,1)
Write-Progress -Activity "Collecting info on $($vm.Name)" -Status "$percentComplete% Complete" -PercentComplete $percentComplete

# Store VM stat info in PSObject

$object = New-Object PSObject

Add-Member -InputObject $object -MemberType NoteProperty -Name VirtualCenter -Value $vm.Uid.Split(":")[0].Split("@")[1]
Add-Member -InputObject $object -MemberType NoteProperty -Name VMName -Value $vm.Name
Add-Member -InputObject $object -MemberType NoteProperty -Name Domain -Value (($vm.Guest.Hostname.Split('.') | Select-Object -Skip 1) -join '.')
Add-Member -InputObject $object -MemberType NoteProperty -Name PowerState -Value $vm.PowerState
Add-Member -InputObject $object -MemberType NoteProperty -Name Uptime -Value (Get-Stat -Entity $VM -Stat sys.osuptime.latest -Realtime -MaxSamples 1 -ErrorAction SilentlyContinue | Select @{N='Uptime (d.hh:mm:ss)';E={[timespan]::FromSeconds($_.value)}})
Add-Member -InputObject $object -MemberType NoteProperty -Name IPAddress -Value ($vm.Guest.IPAddress -join ',')
Add-Member -InputObject $object -MemberType NoteProperty -Name OperatingSystem -Value $vm.Guest.OSFullName
Add-Member -InputObject $object -MemberType NoteProperty -Name CPU -Value $vm.NumCpu
Add-Member -InputObject $object -MemberType NoteProperty -Name MemoryGB -Value $vm.MemoryGB
Add-Member -InputObject $object -MemberType NoteProperty -Name VMToolVersion -Value ($vm.Guest.ToolsVersion)
Add-Member -InputObject $object -MemberType NoteProperty -Name VMversion -Value $vm.hardwareversion
Add-Member -InputObject $object -MemberType NoteProperty -Name Snapshot -Value (&{$script:snaps = Get-Snapshot -VM $vm.Name; $script:snaps.Name -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name SnapshotDate -Value ($script:snaps.Created -join ',')
Add-Member -InputObject $object -MemberType NoteProperty -Name SnapshotSize -Value ([math]::Round($script:snaps.SizeGB -join ',',2))
Add-Member -InputObject $object -MemberType NoteProperty -Name vCluster -Value ($vm | Get-Cluster).Name
Add-Member -InputObject $object -MemberType NoteProperty -Name ESXhost -Value $vm.VMHost
Add-Member -InputObject $object -MemberType NoteProperty -Name ESXVersion -Value $vm.VMHost.version
Add-Member -InputObject $object -MemberType NoteProperty -Name Datastore -Value (&{$DS = Get-Datastore -VM $vm.Name; $DS.Name -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name DatastoreCapacityGB -Value (&{$DS = Get-Datastore -VM $vm.Name; $DS.CapacityGB -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name DatastoreFreespaceGB -Value (&{$DS = Get-Datastore -VM $vm.Name; $DS.FreespaceGB -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name DatastoreNAAID -Value (&{$DS = Get-Datastore -VM $vm.Name; $DS.ExtensionData.Info.Vmfs.Uuid -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name VMFSVersion -Value (&{$DS = Get-Datastore -VM $vm.Name; $DS.ExtensionData.Info.Vmfs.Version -join ','})
Add-Member -InputObject $object -MemberType NoteProperty -Name VMProvisionedSpaceGB -Value ([math]::Round($vm.ProvisionedSpaceGB,0))
Add-Member -InputObject $object -MemberType NoteProperty -Name VMUsedSpaceGB -Value ([math]::Round($vm.UsedSpaceGB, 0))


# Stores the PSObject containing VM stats in to an PSObject array
$results += $object
}

# The Sort-Object is specifically set up so that the Export-Csv and Out-GridView do not truncate the properties of the individual PSObjects in
# the array.

$results | Export-Csv -Path $logFile -NoTypeInformation
Write-Host "Output saved to $logFile"

Disconnect-VIServer -Server * -Confirm:$false -Force | Out-Null

Reply
0 Kudos
LucD
Leadership
Leadership

Did you try the solution discusses in this thread (Start-Job)?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
swarajbrp
Contributor
Contributor

I am not sure exactly where to include this  start-job in the script.

Reply
0 Kudos
LucD
Leadership
Leadership

You can expand your knowledge of background jobs in about_Jobs.

It's not that you would want me to write that script for you.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference