9 Replies Latest reply on Dec 2, 2013 12:41 AM by LucD

    Stats for Each Datastore

    dvnguyen Novice

      I am trying to write a script to determine the average read and write rates, average number of read and write requests, and average read and write latency for each datastore.

       

      I've tried doing it two different ways with 0 as my ending numbers.

       

      $startdate = (Get-Date -Hour 7 -Minute 0 -Second 0).AddDays(-1)

      $enddate = (Get-Date)

       

      $datastores = get-datastore

       

      foreach ($DS in $datastores){

           #1

           $dswriterate = [string]([Math]::Round((($DS.datastore.write.average | Measure-Object Value -Average).Average),2))

       

           #2

           $dswriterate = [string]([Math]::Round((($DS.write.average | Measure-Object Value -Average).Average),2))

      }

       

      I understand that Get-Stat does not work with a datastore as an entity.  It would be great if I could receive help with this!

        • 1. Re: Stats for Each Datastore
          LucD Guru
          Community WarriorsvExpertUser Moderators

          That is correct.

          You could use my Get-Stat2 function to retrieve the available datastore counters.

          See Monitor the size of your vDisks

           

          With the QueryMetrics switch you can discover which metrics are available.

          See Datastore usage statistics

          Blog: http://lucd.info | Twitter: @LucD22 | PowerCLI Reference co-author: http://tinyurl.com/hkn4glz
          • 2. Re: Stats for Each Datastore
            dvnguyen Novice

            Thanks LucD!

            I was able to retrieve the appropriate characteristics using the following format:

             

            foreach ($DS in $datastores){

               $dsdata = (Get-Datastore $DS).Extensiondata

              

               $dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end

            • 3. Re: Stats for Each Datastore
              Kiil Novice
              vExpert

              Hi,

               

              are there any changes in PowerCli and/or vSphere since 2011 that could make some trouble with the get-stat2 function ?  I'm running vSphere 5.0 U2 and PowerCli 5.1

               

              I'm trying to retreive performance counters for each datastore in my environment, and are looking for (maby need to gather info in different ways) :

               

              - ReadLatency (Latest)

              - WriteLatency (Latest)

              - Capacity (Latest)

              - CapacityUsed

              - CapacityFree (calulation from the two above)

              - NumberOfVMs (on the actual datastore)

               

              When running the function with the switches mentioned above ($dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end) it returns this error.

              PowerCLI1.PNG

               

              When trying this command: "get-stat2 -entity $ds -interval "HI2" -stat "datastore.read.average" -queryinstance" I get nothing.

               

              Heres the metrics I retreive when running this command: "get-stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize"

               

              PowerCLI2.PNG

              • 4. Re: Stats for Each Datastore
                LucD Guru
                vExpertUser ModeratorsCommunity Warriors

                The date you pass on the Finish parameter needs to be a DateTime object, you can create one like this for example

                $end = Get-Date -Day 18 -Month 11 -Year 2013

                See if that makes a difference

                Blog: http://lucd.info | Twitter: @LucD22 | PowerCLI Reference co-author: http://tinyurl.com/hkn4glz
                • 5. Re: Stats for Each Datastore
                  Kiil Novice
                  vExpert

                  Hi,

                   

                  Thanks for replying so fast, but that didn't make any differences.  The data format from "get-date" gives the same here...

                   

                  But, it seems that there are some changes in the metrics names.  When adjusting a little on the command the line below, at least, I don't get any error...in fact...I get nothing...

                   

                  get-stat2 -Entity $ds -interval "HI2" -Stat "datastore.numberReadAveraged.average"

                  • 6. Re: Stats for Each Datastore
                    Kiil Novice
                    vExpert

                    Unfortunately, that didn't do the trick.  I've tried to use the "Group.Name.Rollup" as stat in the script, but no luck.  Do you know if there are changes to the metrics that causes this ?  I really need to get the latency numbers logged in a file - could anyone help, please ?

                    • 7. Re: Stats for Each Datastore
                      LucD Guru
                      User ModeratorsvExpertCommunity Warriors

                      No, these metrics should still be there.

                       

                      What object(s) do you have in $ds ?

                      Perhaps you could include the code you are using ?

                      Blog: http://lucd.info | Twitter: @LucD22 | PowerCLI Reference co-author: http://tinyurl.com/hkn4glz
                      • 8. Re: Stats for Each Datastore
                        Kiil Novice
                        vExpert

                        Hi,

                         

                        here's what I've done...see script below:

                         

                        function Get-Stat2 {

                        <#

                        .SYNOPSIS  Retrieve vSphere statistics

                        .DESCRIPTION The function is an alternative to the Get-Stat cmdlet.

                          It's primary use is to provide functionality that is missing

                          from the Get-Stat cmdlet.

                        .NOTES  Author:  Luc Dekens

                        .PARAMETER Entity

                          Specify the VIObject for which you want to retrieve statistics

                          This needs to be an SDK object

                        .PARAMETER Start

                          Start of the interval for which to retrive statistics

                        .PARAMETER Finish

                          End of the interval for which to retrive statistics

                        .PARAMETER Stat

                          The identifiers of the metrics to retrieve

                        .PARAMETER Instance

                          The instance property of the statistics to retrieve

                        .PARAMETER Interval

                          Specify for which interval you want to retrieve statistics.

                          Allowed values are RT, HI1, HI2, HI3 and HI4

                        .PARAMETER MaxSamples

                          The maximum number of samples for each metric

                        .PARAMETER QueryMetrics

                          Switch to indicate that the function should return the available

                          metrics for the Entity specified

                        .PARAMETER QueryInstances

                          Switch to indicate that the function should return the valid instances

                          for a specific Entity and Stat

                        .EXAMPLE

                          PS> Get-Stat2 -Entity $vm.Extensiondata -Stat "cpu.usage.average" -Interval "RT"

                        #>

                         

                          [CmdletBinding()]

                          param (

                          [parameter(Mandatory = $true,  ValueFromPipeline = $true)]

                          [PSObject]$Entity,

                          [DateTime]$Start,

                          [DateTime]$Finish,

                          [String[]]$Stat,

                          [String]$Instance = "",

                          [ValidateSet("RT","HI1","HI2","HI3","HI4")]

                          [String]$Interval = "RT",

                          [int]$MaxSamples,

                          [switch]$QueryMetrics,

                          [switch]$QueryInstances)

                         

                          # Test if entity is valid

                          $EntityType = $Entity.GetType().Name

                         

                          if(!(("HostSystem",

                                "VirtualMachine",

                                "ClusterComputeResource",

                                "Datastore",

                                "ResourcePool") -contains $EntityType)) {

                            Throw "-Entity parameters should be of type HostSystem, VirtualMachine, ClusterComputeResource, Datastore or ResourcePool"

                          }

                         

                          $perfMgr = Get-View (Get-View ServiceInstance).content.perfManager

                         

                          # Create performance counter hashtable

                          $pcTable = New-Object Hashtable

                          $keyTable = New-Object Hashtable

                          foreach($pC in $perfMgr.PerfCounter){

                            if($pC.Level -ne 99){

                              if(!$pctable.containskey($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType)){

                                $pctable.Add(($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType),$pC.Key)

                                $keyTable.Add($pC.Key, $pC)

                              }

                            }

                          }

                         

                          # Test for a valid $Interval

                          if($Interval.ToString().Split(" ").count -gt 1){

                            Throw "Only 1 interval allowed."

                          }

                         

                          $intervalTab = @{"RT"=$null;"HI1"=0;"HI2"=1;"HI3"=2;"HI4"=3}

                          $dsValidIntervals = "HI2","HI3","HI4"

                          $intervalIndex = $intervalTab[$Interval]

                         

                          if($EntityType -ne "datastore"){

                            if($Interval -eq "RT"){

                              $numinterval = 20

                            }

                            else{

                              $numinterval = $perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod

                            }

                          }

                          else{

                            if($dsValidIntervals -contains $Interval){

                              $numinterval = $null

                              if(!$Start){

                                $Start = (Get-Date).AddSeconds($perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod - $perfMgr.HistoricalInterval[$intervalIndex].Length)

                              }

                              if(!$Finish){

                                $Finish = Get-Date

                              }

                            }

                            else{

                              Throw "-Interval parameter $Interval is invalid for datastore metrics."

                            }

                          }

                         

                          # Test if QueryMetrics is given

                          if($QueryMetrics){

                            $metrics = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

                            $metricslist = @()

                            foreach($pmId in $metrics){

                              $pC = $keyTable[$pmId.CounterId]

                              $metricslist += New-Object PSObject -Property @{

                                Group = $pC.GroupInfo.Key

                                Name = $pC.NameInfo.Key

                                Rollup = $pC.RollupType

                                Id = $pC.Key

                                Level = $pC.Level

                                Type = $pC.StatsType

                                Unit = $pC.UnitInfo.Key

                              }

                            }

                            return ($metricslist | Sort-Object -unique -property Group,Name,Rollup)

                          }

                         

                          # Test if start is valid

                          if($Start -ne $null -and $Start -ne ""){

                            if($Start.gettype().name -ne "DateTime") {

                              Throw "-Start parameter should be a DateTime value"

                            }

                          }

                         

                          # Test if finish is valid

                          if($Finish -ne $null -and $Finish -ne ""){

                            if($Finish.gettype().name -ne "DateTime") {

                              Throw "-Start parameter should be a DateTime value"

                            }

                          }

                         

                          # Test start-finish interval

                          if($Start -ne $null -and $Finish -ne $null -and $Start -ge $Finish){

                            Throw "-Start time should be 'older' than -Finish time."

                          }

                         

                          # Test if stat is valid

                          $unitarray = @()

                          $InstancesList = @()

                         

                          foreach($st in $Stat){

                            if($pcTable[$st] -eq $null){

                              Throw "-Stat parameter $st is invalid."

                            }

                            $pcInfo = $perfMgr.QueryPerfCounter($pcTable[$st])

                            $unitarray += $pcInfo[0].UnitInfo.Key

                            $metricId = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

                         

                            # Test if QueryInstances in given

                            if($QueryInstances){

                              $mKey = $pcTable[$st]

                              foreach($metric in $metricId){

                                if($metric.CounterId -eq $mKey){

                                  $InstancesList += New-Object PSObject -Property @{

                                    Stat = $st

                                    Instance = $metric.Instance

                                  }

                                }

                              }

                            }

                            else{

                              # Test if instance is valid

                              $found = $false

                              $validInstances = @()

                              foreach($metric in $metricId){

                                if($metric.CounterId -eq $pcTable[$st]){

                                  if($metric.Instance -eq "") {$cInstance = '""'} else {$cInstance = $metric.Instance}

                                  $validInstances += $cInstance

                                  if($Instance -eq $metric.Instance){$found = $true}

                                }

                              }

                              if(!$found){

                                Throw "-Instance parameter invalid for requested stat: $st.`nValid values are: $validInstances"

                              }

                            }

                          }

                          if($QueryInstances){

                            return $InstancesList

                          }

                         

                          $PQSpec = New-Object VMware.Vim.PerfQuerySpec

                          $PQSpec.entity = $Entity.MoRef

                          $PQSpec.Format = "normal"

                          $PQSpec.IntervalId = $numinterval

                          $PQSpec.MetricId = @()

                          foreach($st in $Stat){

                            $PMId = New-Object VMware.Vim.PerfMetricId

                            $PMId.counterId = $pcTable[$st]

                            if($Instance -ne $null){

                              $PMId.instance = $Instance

                            }

                            $PQSpec.MetricId += $PMId

                          }

                          $PQSpec.StartTime = $Start

                          $PQSpec.EndTime = $Finish

                          if($MaxSamples -eq 0 -or $numinterval -eq 20){

                            $PQSpec.maxSample = $null

                          }

                          else{

                            $PQSpec.MaxSample = $MaxSamples

                          }

                          $Stats = $perfMgr.QueryPerf($PQSpec)

                         

                          # No data available

                          if($Stats[0].Value -eq $null) {return $null}

                         

                          # Extract data to custom object and return as array

                          $data = @()

                          for($i = 0; $i -lt $Stats[0].SampleInfo.Count; $i ++ ){

                            for($j = 0; $j -lt $Stat.Count; $j ++ ){

                              $data += New-Object PSObject -Property @{

                                CounterId = $Stats[0].Value[$j].Id.CounterId

                                CounterName = $Stat[$j]

                                Instance = $Stats[0].Value[$j].Id.Instance

                                Timestamp = $Stats[0].SampleInfo[$i].Timestamp

                                Interval = $Stats[0].SampleInfo[$i].Interval

                                Value = $Stats[0].Value[$j].Value[$i]

                                Unit = $unitarray[$j]

                                Entity = $Entity.Name

                                EntityId = $Entity.MoRef.ToString()

                              }

                            }

                          }

                          if($MaxSamples -eq 0){

                            $data | Sort-Object -Property Timestamp -Descending

                          }

                          else{

                            $data | Sort-Object -Property Timestamp -Descending | select -First $MaxSamples

                          }

                        }

                         

                        $start = Get-Date -Day 27 -Month 11 -Year 2013

                        $end = get-date -Day 28 -Month 11 -Year 2013

                         

                        $ds = (Get-Datastore vcloud01).Extensiondata

                        Get-Stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize  ## OK

                         

                        Get-Stat2 -Entity $ds -Stat "datastore.read.average" -Start $start -Finish $end  ## This doesn't work! See error message

                         

                        This error message appear because it expects an "Interval" switch...like "HI2"...but souldn't this be unnecessary as long as I speicfy from/to dates ?  Anyway, I've tried both ways, and adding the ""-Interval" doesn't give me any latency numbers... :-\

                         

                        Error message:

                        -Interval parameter RT is invalid for datastore metrics.

                        At C:\users\bok\Documents\04. Kundecaser\Conceptos AS\Visma IT\VMware\get-stats2.ps1:101 char:7

                        +       Throw "-Interval parameter $Interval is invalid for datastore metrics."

                        +       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                            + CategoryInfo          : OperationStopped: (-Interval param...astore metrics.:String) [], Run

                           timeException

                            + FullyQualifiedErrorId : -Interval parameter RT is invalid for datastore metrics.


                        Thanks for helping out...really appreciate it!

                        • 9. Re: Stats for Each Datastore
                          LucD Guru
                          Community WarriorsvExpertUser Moderators

                          The default interval is the RealTime interval (RT), but the datastore metrics do not exist in that interval.

                          Add the Interval parameter.

                          Blog: http://lucd.info | Twitter: @LucD22 | PowerCLI Reference co-author: http://tinyurl.com/hkn4glz