I am working with a colleague on a script that will deploy a very large number of VM's from a CSV file. We started with a purely synchronous process that was very slow using a single foreach loop to deploy a VM, configure it, apply a customization spec and start it all based on data from the CSV.
The obvious bottleneck in the process is waiting for the clone operation to complete prior to performing the remaining tasks. Since the synchronous New-VM line was in a foreach loop, it was painfully slow.
We have since modified it to asynchronously deploy the VM's using the foreach loop based the data from the CSV file dumped into an array. Once the foreach loop has kicked off all the clone operations, it finishes and then a task monitoring "while" loop kicks off to then perform the remainder of the tasks. The while loop is based on LucD's code found here: http://www.lucd.info/2010/02/21/about-async-tasks-the-get-task-cmdlet-and-a-hash-table/ (Thanks Luc!).
Now Luc's code used a hash-table so it had to be modified to use the CSV array as best I could. The CSV array contained the information to perform the configuration and customization of the VM's. I really couldn't figure out a way to do this very cleanly so I am looking for suggestions on making this a little better.
Right now I am using a simple counting mechanism to walk through the array to determine if the cloning operation has completed or not. The problem with this is that if there are 100 VM's being cloned, and I am walking through the array one line at a time so it could take a while between successful completion of the clone operation and the monitoring loop actually picking up on it. It would be nice to have a way to identify which line in the array has the required configuration and customization data without walking through it in a loop, rather referencing an element inside the array (TaskID) using some other method (is there one? - still figuring this powershell stuff out).
Here is the modified version of Luc's code:
# $csv is the array in question with the required data.
# Used to count which line in the array
$csvline = 0
# Count all the running tasks to feed into the while loop
$tasks = $csv | %{$_.TaskID} | ?{$_ -match "Task"}
$runningTasks = $tasks | measure | %{$_.count}
while($runningTasks -gt 0){
# Completion Meter
$percomplete = (1 / $runningTasks)*100
Write-Progress -activity 'Waiting for cloning operation to complete' -status 'Please wait...' -percentComplete ($percomplete)
# Here is where it starts to get messy, there has to be a better way than using $csv[$csvline] and walking though
if ((get-task | select id,state | ?{$_.id -eq $csv[$csvline].TaskId}).state -eq "Success"){
Set-VM $csv[$csvline].name -NumCpu $csv[$csvline].vcpu -MemoryMB $csv[$csvline].MemoryMB -Description $csv[$csvline].Notes -Confirm:$false
Get-vm -name $csv[$csvline].name | Get-NetworkAdapter | Set-NetworkAdapter -NetworkName $csv[$csvline].Network -StartConnected:$true -Confirm:$false
Get-VM $csv[$csvline].name | Start-VM -RunAsync -Confirm:$false
$csv[$csvline].TaskId = ""
$runningTasks--
}
elseif ((get-task | select id,state | ?{$_.id -eq $csv[$csvline].TaskId}).state -eq "Error"){
$csv[$csvline].TaskId = ""
$runningTasks--
}
# Increment $csvline
$csvline++
# Wash rinse repeat over and over (not very pretty)
# Reset $CSV array line counter when greater than count of lines (minus 1 because the array/count starts at zero).
if ($csvline -gt ($csv.count - 1)){
$csvline = 0
}
# Slow down the runningTasks loop since we are waiting for cloning operations to complete.
# IMPACT: If you deploy 100 VM's it could take up to 200 seconds AFTER a VM is finished cloning before being noticed by the while loop
Start-Sleep -Seconds 2
}
It would be nice to create a Get-Task monitoring loop that doesn't have to walk through the array. But pull any "Sucess"ful tasknames and figure out which line in the $csv array it belongs to (using the TaskId element).
Is there a way to do this other than nesting a foreach ($line in $csv) inside the Get-Task monitoring loop and compare TaskId values? (That might be quicker regardless - haven't tested yet).
Hopefully I explained things clearly enough.
Thanks for your time.
Andy