VMware Cloud Community
electricd7
Contributor
Contributor

Need help generating virtualdisk statistics for all machines

Hello,


I would like to come up with a script that will monitor all disks on all powered on machines for read and write latency as well as number of writes.  I am basically trying to create a tool that will help me better choose which machines might benefit from virtual flash read-cache resources.  I have set my statistic level to 2 for all counters except 5min which I set to 4.  I am basically wanting to run a script every X mins or hours which will collect these stats per disk and store them in a CSV until i get a a DB setup.  I found some code that was written by LucD (i think) that I think should work for this report, but its not working in my environment.  The counters are coming back as null.  What am I doing wrong?  I am able to go into vcenter and look at the performance tabs and see this data for at least the past day, so i know at least some data exists.  I have copied my code below:


# Main Variables

$sVCentre = "virtualcenter"

$strVMWildCard = "*"

$strCSVName = "Stats-AvgVMDiskWriteStatsWorkingday"

$strCSVLocation = "c:\temp\"

$metrics = "virtualDisk.totalWriteLatency.average","virtualDisk.totalReadLatency.average",

    "virtualDisk.numberReadAveraged.average","virtualDisk.numberWriteAveraged.average",

    "virtualDisk.read.average","virtualDisk.write.average"


#define the start and finish times for a working day.

$today9am = (Get-Date -Hour 9 -Minute 0 -Second 0)

$today5pm = (Get-Date -Hour 17 -Minute 0 -Second 0)

$intStartDay = -1

$intEndDay = -1

#Begin Script

#Connect to VC

Connect-VIServer $sVCentre -ea silentlycontinue

$arrVMs = Get-Cluster "Prod 1" | Get-VM | where-object {$_.Name -like $strVMWildCard} | Sort Name

$stats = Get-Stat -Entity $arrVMs -Stat $metrics -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)


#group the data and collate the stats into averages for the day.

$groups = $stats | Group-Object -Property {$_.Entity, $_.MetricId, $_.Instance}

$report = $groups | % {

    New-Object PSObject -Property @{

        Description = $_.Group[0].Description

        Entity = $_.Group[0].Entity

        EntityId = $_.Group[0].EntityId

        Instance = $_.Group[0].Instance

        MetricId = $_.Group[0].MetricId

        Timestamp = $_.Group[0].Timestamp.Date.AddHours($_.Group[0].Timestamp.Hour)

        Unit = $_.Group[0].Unit

        Value = [math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 2)

    }

}

#Exporting the report to a CSV file.

$strCSVSuffix = (get-date).toString('yyyyMMddhhmm')

$strCSVFile = $strCSVLocation + $strCSVName + "_" + $strCSVSuffix + ".csv"

$report | Export-Csv $strCSVfile -NoTypeInformation -UseCulture

0 Kudos
5 Replies
electricd7
Contributor
Contributor



If I add " -IntervalMins 5" to the end of the $stats line, I get data but the script never completes ( i have waited 2 hours) so I am guessing that is not exactly what I want.

0 Kudos
electricd7
Contributor
Contributor

Ok I have verified that adding the "-IntervalMins 5" generates the data I want when I do them one at a time, however, the script as its written with that switch has now been running for 8 hours at 50% CPU and still hasn't finished.  An example of the data I want to get when I run this one at a time looks like the following.  Is there an easier way to get this than to get to 5 minute stats for each counter and then use PowerShell to group them as I have done above?

MetricIdTimestampUnitEntityIdDescriptionInstanceValueEntity
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:00.52Ffcobdb
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:10Ffcobdb
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:2429.75Ffcobdb
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:30Ffcobdb
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:40.02Ffcobdb
virtualdisk.numberreadaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of read commands issued per second to the virtual disk during the collection intervalscsi0:50Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:02.92Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:10Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:20.86Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:32.68Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:40Ffcobdb
virtualdisk.numberwriteaveraged.average3/4/14 23:00numberVirtualMachine-vm-37165Average number of write commands issued per second to the virtual disk during the collection intervalscsi0:50.01Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual disk26246.61Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:029.84Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:10Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:226215.78Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:30.02Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:40.07Ffcobdb
virtualdisk.read.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of reading data from the virtual diskscsi0:50.22Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual disk89.51Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:063.63Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:10.1Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:210.93Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:32.74Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:46.62Ffcobdb
virtualdisk.write.average3/4/14 23:00KBpsVirtualMachine-vm-37165Rate of writing data to the virtual diskscsi0:51.8Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:010.51Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:10Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:29.81Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:30.08Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:40.18Ffcobdb
virtualdisk.totalreadlatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a read from the virtual disk takesscsi0:51.12Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:05.14Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:10.58Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:21.65Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:32.04Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:46.94Ffcobdb
virtualdisk.totalwritelatency.average3/4/14 23:00millisecondVirtualMachine-vm-37165The average time a write to the virtual disk takesscsi0:51.9Ffcobdb
0 Kudos
electricd7
Contributor
Contributor



OK so it turns out that the REALLY slow portion of this script turns out to be the grouping portion where i take the stats and group them by entity, disk, and metric.  I am trying to derive the average for each of the disk metric types over a 24 hour period.  This report pulls up really quickly in the Performance tab in vCenter so surely this value is created somewhere as part of the Daily roll up job.  I am just having a hard time getting that value per disk using the Get-Stat command.  Help?

0 Kudos
LucD
Leadership
Leadership

There are many factors that might influence the required time to run this script; the number of metrics returned by the Get-Stat cmdlet, the number of VMs for which you run the report.

In the vSphere client you probably get the metrics for 1 specific VM.

There are some options to explore and possibly make this script run a bit faster faster.

  • If the station on which you run the script has sufficient resources, you might consider running several instances of the script in parallel. Use the Start-Job cmdlet for that.
  • Instead of creating a row per metric, consider makeing a row per vDisk which has all the metrics. This will allow you to run the Group-Object on 1 less property for the grouping
  • Consider using my Get-Stat2 function, which normally runs faster than the Get-Stat cmdlet


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
electricd7
Contributor
Contributor

Thanks for the advice LucD.  I ended up putting the raw stats into a second SQL database that I could properly index and do all of my grouping/reporting out of that database.

0 Kudos