stuartgmilton
Enthusiast
Enthusiast

Script Performance Optimization request

Jump to solution

Hi Guys,

I have literally just started scripting VMWare, I think this is my 3rd day in total, so apologies if this is a complete pile of junk, but it works.

I have the below script to run for around 700 VMs, and at the moment, it is taking around 14 seconds per VM, so around a few hours total.

$allvms = @()

$vms = Get-Vm | where {$_.powerstate -eq “PoweredOn”}

foreach($vm in $vms)

{

  $vmstat = "" | Select VmName, VmTools, MemAvg, CPUAvg,  diskreadAvg, diskwriteAvg

  $vmstat.VmName = $vm.name

  $vmstat.VmTools = $vm.Guest.toolsVersionStatus

 

  $statcpu = Get-Stat -Entity ($vm)-start (get-date).AddDays(-7) -Finish (Get-Date)-MaxSamples 10000 -stat cpu.usage.average

  $statmem = Get-Stat -Entity ($vm)-start (get-date).AddDays(-7) -Finish (Get-Date)-MaxSamples 10000 -stat mem.usage.average

  $statdiskread = Get-Stat -Entity ($vm)-start (get-date).AddDays(-7) -Finish (Get-Date)-MaxSamples 10000 -stat disk.read.average

  $statdiskwrite = Get-Stat -Entity ($vm)-start (get-date).AddDays(-7) -Finish (Get-Date)-MaxSamples 10000 -stat disk.write.average

  $cpu = $statcpu | Measure-Object -Property value -Average #-Maximum -Minimum

  $mem = $statmem | Measure-Object -Property value -Average #-Maximum -Minimum

  $diskread = $statdiskread | Measure-Object -Property value -Average #-Maximum -Minimum

  $diskwrite = $statdiskwrite | Measure-Object -Property value -Average #-Maximum -Minimum

  $vmstat.CPUAvg = $cpu.Average

  $vmstat.MemAvg = $mem.Average

  $vmstat.diskreadAvg = $diskread.Average

  $vmstat.diskwriteAvg = $diskwrite.Average

  $allvms += $vmstat

}

$allvms | Select VmName, VmTools, MemAvg, CPUAvg,  diskreadAvg,  diskwriteAvg | Export-Csv "c:\VMs.csv" -noTypeInformation


Is it possible to speed this up significantly??

Thanks for any help,

Stuart

0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership

I suspect the problem is due to the back-tick, followed by 1 or more blanks, you have on line 4.

Try like this (for me this works)

$metrics = "cpu.usage.average","mem.usage.average","disk.read.average","disk.write.average" 
$vms = Get-VM | where {$_.name -like "Git*"}
$start = (Get-Date).AddDays(-7)
$allvms = Get-Stat -Entity $vms -Stat $metrics -MaxSamples ([int]::MaxValue) -Start $start |
Group-Object -Property EntityId | %{
 
New-Object PSObject -Property @{
   
Name = $_.Group[0].Entity.Name
   
statCPU = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statMem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statDiskRead = $_.Group | where {$_.MetricId -eq "disk.read.average" -and $_.Instance -eq ""} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statDiskWrite = $_.Group | where {$_.MetricId -eq "disk.write.average" -and $_.Instance -eq ""} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
  }
}
$allvms | Sort-Object Name | Export-Csv "C:\VMs.csv" -NoTypeInformation

I added a filter on the Instance in the returned metrics. For these disk metrics there will be multiple instances (one per vDisk and one aggregate), I only kept the aggregate value.

See PowerCLI & vSphere statistics – Part 3 – Instances for some more info.

.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

0 Kudos
12 Replies
LucD
Leadership
Leadership

The first thing I would get rid of are the multiple calls to Get-Stat.

You can reduce that to 1 Get-Stat call with all the counters in 1 go and that for all the VMs together.

Then use the Group-Object cmdlet to collect your returned metrics together.

See for example Re: Powercli script mem, cpu, disk, uptime or Re: CPU Ready Value


You could also consider using my Get-Stat2 function instead of Get-Stat.

Most of the time Get-Stat2 is faster.

See my Dutch VMUG: The Statistics Reporting Session post.

Let me know if you need more help.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

stuartgmilton
Enthusiast
Enthusiast

Thanks Lucd,

Will give this a go first thing 2moro and let you know how it goes.

Many Thanks,
Stuart

0 Kudos
stuartgmilton
Enthusiast
Enthusiast

Hi,

So I've changed the script a bit as per your suggestion, but now no data is being displayed in the statCPU or statMem column - Where have I been stupid!?!?

$metrics = "cpu.usage.average", "mem.usage.average"#, "disk.read.average", "disk.write.average"

$vms = Get-VM | where {$_.name -like “Git*”}

$start = (Get-Date).AddDays(-7)

$allvms = Get-Stat -Entity $vms -Stat $metrics -MaxSamples 10000  -Start $start | `

Group-Object -Property EntityId | %{

  New-Object PSObject -Property @{

    Name = $_.Group[0].Entity.Name

    statCPU = ($_.Group | where {$_.MetricId -eq "cpu.usage.average"}).Value

    statMem = ($_.Group | where {$_.MetricId -eq "mem.usage.average"}).Value

  }

}

$allvms | Sort-Object Name | Export-Csv "C:\VMs.csv" -NoTypeInformation

Cheers,
Stuart

0 Kudos
LucD
Leadership
Leadership

Try like this

$metrics = "cpu.usage.average", "mem.usage.average"#, "disk.read.average", "disk.write.average"
$vms = Get-VM | where {$_.name -like Git*}
$start = (Get-Date).AddDays(-7)
$allvms = Get-Stat -Entity $vms -Stat $metrics -MaxSamples ([int]::MaxValue-Start $start | `
Group-Object -Property EntityId | %{
 
New-Object PSObject -Property @{
   
Name = $_.Group[0].Entity.Name
   
statCPU = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} |
     
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statMem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} |
     
Measure-Object -Property Value -Average | Select -ExpandProperty Average
  }
}
$allvms | Sort-Object Name | Export-Csv "C:\VMs.csv" -NoTypeInformation


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

stuartgmilton
Enthusiast
Enthusiast

Ahh superb, thankyou.

But weirdly, when I uncomment my final 2 stats and add in the required lines lower down, now they don't return info - but they are available metrics.

Sorry to take up so much of your time...

$metrics = "cpu.usage.average", "mem.usage.average", "disk.read.average", "disk.write.average"

$vms = Get-VM | where {$_.name -like “Git*”}

$start = (Get-Date).AddDays(-7)

$allvms = Get-Stat -Entity $vms -Stat $metrics -MaxSamples ([int]::MaxValue)  -Start $start | `

Group-Object -Property EntityId | %{

  New-Object PSObject -Property @{

    Name = $_.Group[0].Entity.Name

    statCPU = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} |

      Measure-Object -Property Value -Average | Select -ExpandProperty Average

    statMem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} |

      Measure-Object -Property Value -Average | Select -ExpandProperty Average

    statDiskRead = $_.Group | where {$_.MetricId -eq "disk.read.average"} |

      Measure-Object -Property Value -Average | Select -ExpandProperty Average

    statDiskWrite = $_.Group | where {$_.MetricId -eq "disk.write.average"} |

      Measure-Object -Property Value -Average | Select -ExpandProperty Average

  }

}

$allvms | Sort-Object Name | Export-Csv "C:\VMs.csv" -NoTypeInformation

0 Kudos
LucD
Leadership
Leadership

I suspect the problem is due to the back-tick, followed by 1 or more blanks, you have on line 4.

Try like this (for me this works)

$metrics = "cpu.usage.average","mem.usage.average","disk.read.average","disk.write.average" 
$vms = Get-VM | where {$_.name -like "Git*"}
$start = (Get-Date).AddDays(-7)
$allvms = Get-Stat -Entity $vms -Stat $metrics -MaxSamples ([int]::MaxValue) -Start $start |
Group-Object -Property EntityId | %{
 
New-Object PSObject -Property @{
   
Name = $_.Group[0].Entity.Name
   
statCPU = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statMem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statDiskRead = $_.Group | where {$_.MetricId -eq "disk.read.average" -and $_.Instance -eq ""} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
   
statDiskWrite = $_.Group | where {$_.MetricId -eq "disk.write.average" -and $_.Instance -eq ""} |
   
Measure-Object -Property Value -Average | Select -ExpandProperty Average
  }
}
$allvms | Sort-Object Name | Export-Csv "C:\VMs.csv" -NoTypeInformation

I added a filter on the Instance in the returned metrics. For these disk metrics there will be multiple instances (one per vDisk and one aggregate), I only kept the aggregate value.

See PowerCLI & vSphere statistics – Part 3 – Instances for some more info.

.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
stuartgmilton
Enthusiast
Enthusiast

Unfortunately, those last 2 metrics are still being an issue.  Still no data returned.  I have also run the script for 100 of my VMs to see if it was an issue with these "Git*" VMs, but none of them return any information.

Just to confirm, that metrics are available for these VMs.

Confused...

0 Kudos
LucD
Leadership
Leadership

As a quick check, do you see data in the vSphere client under the Performance tab for any of these VM.

Could also be caused by the aggregation job that might have a problem.

Do these jobs run on the DB server where the vCenter DB is ?

Can you try the same script, but with a shorter interval, for example for the last hour.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
stuartgmilton
Enthusiast
Enthusiast

Hi,

So it seems that if a VM if less than the report time period, nothing is returned.

For instance if I only return our Greenhopper & Git VMs, there is data for the Greenhopper boxes but not the Git boxes.

I think I'll limit the report to VMs which are older than 3 months old, as I'm really trying to find VMs which are inactive.

Many Thanks for all your help,

Stuart

0 Kudos
LucD
Leadership
Leadership

That is correct, it would even be advised to add '-ErrorAction SilentlyContinue' on the Get-Stat cmdlet.

Otherwise the cmdlet might just stop if it encounters a VM for which there is no data present.

But you should have seen an error message for that.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
stuartgmilton
Enthusiast
Enthusiast

Hi Luc,

Just for info - these changes reduced my script duration by 50%.

I'm now going to try and your Get-Stat2.

Cheers,
Stuart

0 Kudos
LucD
Leadership
Leadership

Note that some of the parameters on Get-Stat2 are different from the ones on Get-Stat.

On the Entity parameter, my Get-Stat2 expects vSphere objects instead of .Net objects.

In other words

$vms = Get-View -ViewType VirtualMachine -Filter @{"Name"="^Git"}

Get-Stat2 -Entity $vms ....

or

$vms = Get-VM -Name "Git*" | %{$_.ExtensionData}

Get-Stat2 -Entity $vms ....


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos