Automation

 View Only
  • 1.  Stats for Each Datastore

    Posted Aug 21, 2013 06:36 AM

    I am trying to write a script to determine the average read and write rates, average number of read and write requests, and average read and write latency for each datastore.

    I've tried doing it two different ways with 0 as my ending numbers.

    $startdate = (Get-Date -Hour 7 -Minute 0 -Second 0).AddDays(-1)

    $enddate = (Get-Date)

    $datastores = get-datastore

    foreach ($DS in $datastores){

         #1

         $dswriterate = [string]([Math]::Round((($DS.datastore.write.average | Measure-Object Value -Average).Average),2))

         #2

         $dswriterate = [string]([Math]::Round((($DS.write.average | Measure-Object Value -Average).Average),2))

    }

    I understand that Get-Stat does not work with a datastore as an entity.  It would be great if I could receive help with this!



  • 2.  RE: Stats for Each Datastore
    Best Answer

    Posted Aug 21, 2013 09:14 AM

    That is correct.

    You could use my Get-Stat2 function to retrieve the available datastore counters.

    See Monitor the size of your vDisks

    With the QueryMetrics switch you can discover which metrics are available.

    See Datastore usage statistics



  • 3.  RE: Stats for Each Datastore

    Posted Aug 21, 2013 02:44 PM

    Thanks LucD!

    I was able to retrieve the appropriate characteristics using the following format:

    foreach ($DS in $datastores){

       $dsdata = (Get-Datastore $DS).Extensiondata

      

       $dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end



  • 4.  RE: Stats for Each Datastore

    Posted Nov 19, 2013 02:08 PM

    Hi,

    are there any changes in PowerCli and/or vSphere since 2011 that could make some trouble with the get-stat2 function ?  I'm running vSphere 5.0 U2 and PowerCli 5.1

    I'm trying to retreive performance counters for each datastore in my environment, and are looking for (maby need to gather info in different ways) :

    - ReadLatency (Latest)

    - WriteLatency (Latest)

    - Capacity (Latest)

    - CapacityUsed

    - CapacityFree (calulation from the two above)

    - NumberOfVMs (on the actual datastore)

    When running the function with the switches mentioned above ($dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end) it returns this error.

    When trying this command: "get-stat2 -entity $ds -interval "HI2" -stat "datastore.read.average" -queryinstance" I get nothing.

    Heres the metrics I retreive when running this command: "get-stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize"



  • 5.  RE: Stats for Each Datastore

    Posted Nov 19, 2013 04:19 PM

    The date you pass on the Finish parameter needs to be a DateTime object, you can create one like this for example

    $end = Get-Date -Day 18 -Month 11 -Year 2013

    See if that makes a difference



  • 6.  RE: Stats for Each Datastore

    Posted Nov 20, 2013 02:12 PM

    Hi,

    Thanks for replying so fast, but that didn't make any differences.  The data format from "get-date" gives the same here...

    But, it seems that there are some changes in the metrics names.  When adjusting a little on the command the line below, at least, I don't get any error...in fact...I get nothing...

    get-stat2 -Entity $ds -interval "HI2" -Stat "datastore.numberReadAveraged.average"



  • 7.  RE: Stats for Each Datastore

    Posted Nov 29, 2013 08:07 AM

    Unfortunately, that didn't do the trick.  I've tried to use the "Group.Name.Rollup" as stat in the script, but no luck.  Do you know if there are changes to the metrics that causes this ?  I really need to get the latency numbers logged in a file - could anyone help, please ?



  • 8.  RE: Stats for Each Datastore

    Posted Nov 29, 2013 08:26 AM

    No, these metrics should still be there.

    What object(s) do you have in $ds ?

    Perhaps you could include the code you are using ?



  • 9.  RE: Stats for Each Datastore

    Posted Dec 02, 2013 07:33 AM

    Hi,

    here's what I've done...see script below:

    function Get-Stat2 {

    <#

    .SYNOPSIS  Retrieve vSphere statistics

    .DESCRIPTION The function is an alternative to the Get-Stat cmdlet.

      It's primary use is to provide functionality that is missing

      from the Get-Stat cmdlet.

    .NOTES  Author:  Luc Dekens

    .PARAMETER Entity

      Specify the VIObject for which you want to retrieve statistics

      This needs to be an SDK object

    .PARAMETER Start

      Start of the interval for which to retrive statistics

    .PARAMETER Finish

      End of the interval for which to retrive statistics

    .PARAMETER Stat

      The identifiers of the metrics to retrieve

    .PARAMETER Instance

      The instance property of the statistics to retrieve

    .PARAMETER Interval

      Specify for which interval you want to retrieve statistics.

      Allowed values are RT, HI1, HI2, HI3 and HI4

    .PARAMETER MaxSamples

      The maximum number of samples for each metric

    .PARAMETER QueryMetrics

      Switch to indicate that the function should return the available

      metrics for the Entity specified

    .PARAMETER QueryInstances

      Switch to indicate that the function should return the valid instances

      for a specific Entity and Stat

    .EXAMPLE

      PS> Get-Stat2 -Entity $vm.Extensiondata -Stat "cpu.usage.average" -Interval "RT"

    #>

      [CmdletBinding()]

      param (

      [parameter(Mandatory = $true,  ValueFromPipeline = $true)]

      [PSObject]$Entity,

      [DateTime]$Start,

      [DateTime]$Finish,

      [String[]]$Stat,

      [String]$Instance = "",

      [ValidateSet("RT","HI1","HI2","HI3","HI4")]

      [String]$Interval = "RT",

      [int]$MaxSamples,

      [switch]$QueryMetrics,

      [switch]$QueryInstances)

      # Test if entity is valid

      $EntityType = $Entity.GetType().Name

      if(!(("HostSystem",

            "VirtualMachine",

            "ClusterComputeResource",

            "Datastore",

            "ResourcePool") -contains $EntityType)) {

        Throw "-Entity parameters should be of type HostSystem, VirtualMachine, ClusterComputeResource, Datastore or ResourcePool"

      }

      $perfMgr = Get-View (Get-View ServiceInstance).content.perfManager

      # Create performance counter hashtable

      $pcTable = New-Object Hashtable

      $keyTable = New-Object Hashtable

      foreach($pC in $perfMgr.PerfCounter){

        if($pC.Level -ne 99){

          if(!$pctable.containskey($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType)){

            $pctable.Add(($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType),$pC.Key)

            $keyTable.Add($pC.Key, $pC)

          }

        }

      }

      # Test for a valid $Interval

      if($Interval.ToString().Split(" ").count -gt 1){

        Throw "Only 1 interval allowed."

      }

      $intervalTab = @{"RT"=$null;"HI1"=0;"HI2"=1;"HI3"=2;"HI4"=3}

      $dsValidIntervals = "HI2","HI3","HI4"

      $intervalIndex = $intervalTab[$Interval]

      if($EntityType -ne "datastore"){

        if($Interval -eq "RT"){

          $numinterval = 20

        }

        else{

          $numinterval = $perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod

        }

      }

      else{

        if($dsValidIntervals -contains $Interval){

          $numinterval = $null

          if(!$Start){

            $Start = (Get-Date).AddSeconds($perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod - $perfMgr.HistoricalInterval[$intervalIndex].Length)

          }

          if(!$Finish){

            $Finish = Get-Date

          }

        }

        else{

          Throw "-Interval parameter $Interval is invalid for datastore metrics."

        }

      }

      # Test if QueryMetrics is given

      if($QueryMetrics){

        $metrics = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

        $metricslist = @()

        foreach($pmId in $metrics){

          $pC = $keyTable[$pmId.CounterId]

          $metricslist += New-Object PSObject -Property @{

            Group = $pC.GroupInfo.Key

            Name = $pC.NameInfo.Key

            Rollup = $pC.RollupType

            Id = $pC.Key

            Level = $pC.Level

            Type = $pC.StatsType

            Unit = $pC.UnitInfo.Key

          }

        }

        return ($metricslist | Sort-Object -unique -property Group,Name,Rollup)

      }

      # Test if start is valid

      if($Start -ne $null -and $Start -ne ""){

        if($Start.gettype().name -ne "DateTime") {

          Throw "-Start parameter should be a DateTime value"

        }

      }

      # Test if finish is valid

      if($Finish -ne $null -and $Finish -ne ""){

        if($Finish.gettype().name -ne "DateTime") {

          Throw "-Start parameter should be a DateTime value"

        }

      }

      # Test start-finish interval

      if($Start -ne $null -and $Finish -ne $null -and $Start -ge $Finish){

        Throw "-Start time should be 'older' than -Finish time."

      }

      # Test if stat is valid

      $unitarray = @()

      $InstancesList = @()

      foreach($st in $Stat){

        if($pcTable[$st] -eq $null){

          Throw "-Stat parameter $st is invalid."

        }

        $pcInfo = $perfMgr.QueryPerfCounter($pcTable[$st])

        $unitarray += $pcInfo[0].UnitInfo.Key

        $metricId = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

        # Test if QueryInstances in given

        if($QueryInstances){

          $mKey = $pcTable[$st]

          foreach($metric in $metricId){

            if($metric.CounterId -eq $mKey){

              $InstancesList += New-Object PSObject -Property @{

                Stat = $st

                Instance = $metric.Instance

              }

            }

          }

        }

        else{

          # Test if instance is valid

          $found = $false

          $validInstances = @()

          foreach($metric in $metricId){

            if($metric.CounterId -eq $pcTable[$st]){

              if($metric.Instance -eq "") {$cInstance = '""'} else {$cInstance = $metric.Instance}

              $validInstances += $cInstance

              if($Instance -eq $metric.Instance){$found = $true}

            }

          }

          if(!$found){

            Throw "-Instance parameter invalid for requested stat: $st.`nValid values are: $validInstances"

          }

        }

      }

      if($QueryInstances){

        return $InstancesList

      }

      $PQSpec = New-Object VMware.Vim.PerfQuerySpec

      $PQSpec.entity = $Entity.MoRef

      $PQSpec.Format = "normal"

      $PQSpec.IntervalId = $numinterval

      $PQSpec.MetricId = @()

      foreach($st in $Stat){

        $PMId = New-Object VMware.Vim.PerfMetricId

        $PMId.counterId = $pcTable[$st]

        if($Instance -ne $null){

          $PMId.instance = $Instance

        }

        $PQSpec.MetricId += $PMId

      }

      $PQSpec.StartTime = $Start

      $PQSpec.EndTime = $Finish

      if($MaxSamples -eq 0 -or $numinterval -eq 20){

        $PQSpec.maxSample = $null

      }

      else{

        $PQSpec.MaxSample = $MaxSamples

      }

      $Stats = $perfMgr.QueryPerf($PQSpec)

      # No data available

      if($Stats[0].Value -eq $null) {return $null}

      # Extract data to custom object and return as array

      $data = @()

      for($i = 0; $i -lt $Stats[0].SampleInfo.Count; $i ++ ){

        for($j = 0; $j -lt $Stat.Count; $j ++ ){

          $data += New-Object PSObject -Property @{

            CounterId = $Stats[0].Value[$j].Id.CounterId

            CounterName = $Stat[$j]

            Instance = $Stats[0].Value[$j].Id.Instance

            Timestamp = $Stats[0].SampleInfo[$i].Timestamp

            Interval = $Stats[0].SampleInfo[$i].Interval

            Value = $Stats[0].Value[$j].Value[$i]

            Unit = $unitarray[$j]

            Entity = $Entity.Name

            EntityId = $Entity.MoRef.ToString()

          }

        }

      }

      if($MaxSamples -eq 0){

        $data | Sort-Object -Property Timestamp -Descending

      }

      else{

        $data | Sort-Object -Property Timestamp -Descending | select -First $MaxSamples

      }

    }

    $start = Get-Date -Day 27 -Month 11 -Year 2013

    $end = get-date -Day 28 -Month 11 -Year 2013

    $ds = (Get-Datastore vcloud01).Extensiondata

    Get-Stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize  ## OK

    Get-Stat2 -Entity $ds -Stat "datastore.read.average" -Start $start -Finish $end  ## This doesn't work! See error message

    This error message appear because it expects an "Interval" switch...like "HI2"...but souldn't this be unnecessary as long as I speicfy from/to dates ?  Anyway, I've tried both ways, and adding the ""-Interval" doesn't give me any latency numbers... :-\

    Error message:

    -Interval parameter RT is invalid for datastore metrics.

    At C:\users\bok\Documents\04. Kundecaser\Conceptos AS\Visma IT\VMware\get-stats2.ps1:101 char:7

    +       Throw "-Interval parameter $Interval is invalid for datastore metrics."

    +       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

        + CategoryInfo          : OperationStopped: (-Interval param...astore metrics.:String) [], Run

       timeException

        + FullyQualifiedErrorId : -Interval parameter RT is invalid for datastore metrics.


    Thanks for helping out...really appreciate it!



  • 10.  RE: Stats for Each Datastore

    Posted Dec 02, 2013 08:41 AM

    The default interval is the RealTime interval (RT), but the datastore metrics do not exist in that interval.

    Add the Interval parameter.