VMware Cloud Community
dvnguyen
Contributor
Contributor
Jump to solution

Stats for Each Datastore

I am trying to write a script to determine the average read and write rates, average number of read and write requests, and average read and write latency for each datastore.

I've tried doing it two different ways with 0 as my ending numbers.

$startdate = (Get-Date -Hour 7 -Minute 0 -Second 0).AddDays(-1)

$enddate = (Get-Date)

$datastores = get-datastore

foreach ($DS in $datastores){

     #1

     $dswriterate = [string]([Math]::Round((($DS.datastore.write.average | Measure-Object Value -Average).Average),2))

     #2

     $dswriterate = [string]([Math]::Round((($DS.write.average | Measure-Object Value -Average).Average),2))

}

I understand that Get-Stat does not work with a datastore as an entity.  It would be great if I could receive help with this!

Tags (1)
Reply
0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership
Jump to solution

That is correct.

You could use my Get-Stat2 function to retrieve the available datastore counters.

See Monitor the size of your vDisks

With the QueryMetrics switch you can discover which metrics are available.

See Datastore usage statistics


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

Reply
0 Kudos
9 Replies
LucD
Leadership
Leadership
Jump to solution

That is correct.

You could use my Get-Stat2 function to retrieve the available datastore counters.

See Monitor the size of your vDisks

With the QueryMetrics switch you can discover which metrics are available.

See Datastore usage statistics


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
dvnguyen
Contributor
Contributor
Jump to solution

Thanks LucD!

I was able to retrieve the appropriate characteristics using the following format:

foreach ($DS in $datastores){

   $dsdata = (Get-Datastore $DS).Extensiondata

  

   $dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end

Reply
0 Kudos
Kiil
Contributor
Contributor
Jump to solution

Hi,

are there any changes in PowerCli and/or vSphere since 2011 that could make some trouble with the get-stat2 function ?  I'm running vSphere 5.0 U2 and PowerCli 5.1

I'm trying to retreive performance counters for each datastore in my environment, and are looking for (maby need to gather info in different ways) :

- ReadLatency (Latest)

- WriteLatency (Latest)

- Capacity (Latest)

- CapacityUsed

- CapacityFree (calulation from the two above)

- NumberOfVMs (on the actual datastore)

When running the function with the switches mentioned above ($dsreadrate = Get-Stat2 -Entity $dsdata -Stat "datastore.read.average" -Start $start -Finish $end) it returns this error.

PowerCLI1.PNG

When trying this command: "get-stat2 -entity $ds -interval "HI2" -stat "datastore.read.average" -queryinstance" I get nothing.

Heres the metrics I retreive when running this command: "get-stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize"

PowerCLI2.PNG

Bjørn-Ove Kiil
Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

The date you pass on the Finish parameter needs to be a DateTime object, you can create one like this for example

$end = Get-Date -Day 18 -Month 11 -Year 2013

See if that makes a difference


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
Kiil
Contributor
Contributor
Jump to solution

Hi,

Thanks for replying so fast, but that didn't make any differences.  The data format from "get-date" gives the same here...

But, it seems that there are some changes in the metrics names.  When adjusting a little on the command the line below, at least, I don't get any error...in fact...I get nothing...

get-stat2 -Entity $ds -interval "HI2" -Stat "datastore.numberReadAveraged.average"

Bjørn-Ove Kiil
Reply
0 Kudos
Kiil
Contributor
Contributor
Jump to solution

Unfortunately, that didn't do the trick.  I've tried to use the "Group.Name.Rollup" as stat in the script, but no luck.  Do you know if there are changes to the metrics that causes this ?  I really need to get the latency numbers logged in a file - could anyone help, please ?

Bjørn-Ove Kiil
Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

No, these metrics should still be there.

What object(s) do you have in $ds ?

Perhaps you could include the code you are using ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
Kiil
Contributor
Contributor
Jump to solution

Hi,

here's what I've done...see script below:

function Get-Stat2 {

<#

.SYNOPSIS  Retrieve vSphere statistics

.DESCRIPTION The function is an alternative to the Get-Stat cmdlet.

  It's primary use is to provide functionality that is missing

  from the Get-Stat cmdlet.

.NOTES  Author:  Luc Dekens

.PARAMETER Entity

  Specify the VIObject for which you want to retrieve statistics

  This needs to be an SDK object

.PARAMETER Start

  Start of the interval for which to retrive statistics

.PARAMETER Finish

  End of the interval for which to retrive statistics

.PARAMETER Stat

  The identifiers of the metrics to retrieve

.PARAMETER Instance

  The instance property of the statistics to retrieve

.PARAMETER Interval

  Specify for which interval you want to retrieve statistics.

  Allowed values are RT, HI1, HI2, HI3 and HI4

.PARAMETER MaxSamples

  The maximum number of samples for each metric

.PARAMETER QueryMetrics

  Switch to indicate that the function should return the available

  metrics for the Entity specified

.PARAMETER QueryInstances

  Switch to indicate that the function should return the valid instances

  for a specific Entity and Stat

.EXAMPLE

  PS> Get-Stat2 -Entity $vm.Extensiondata -Stat "cpu.usage.average" -Interval "RT"

#>

  [CmdletBinding()]

  param (

  [parameter(Mandatory = $true,  ValueFromPipeline = $true)]

  [PSObject]$Entity,

  [DateTime]$Start,

  [DateTime]$Finish,

  [String[]]$Stat,

  [String]$Instance = "",

  [ValidateSet("RT","HI1","HI2","HI3","HI4")]

  [String]$Interval = "RT",

  [int]$MaxSamples,

  [switch]$QueryMetrics,

  [switch]$QueryInstances)

  # Test if entity is valid

  $EntityType = $Entity.GetType().Name

  if(!(("HostSystem",

        "VirtualMachine",

        "ClusterComputeResource",

        "Datastore",

        "ResourcePool") -contains $EntityType)) {

    Throw "-Entity parameters should be of type HostSystem, VirtualMachine, ClusterComputeResource, Datastore or ResourcePool"

  }

  $perfMgr = Get-View (Get-View ServiceInstance).content.perfManager

  # Create performance counter hashtable

  $pcTable = New-Object Hashtable

  $keyTable = New-Object Hashtable

  foreach($pC in $perfMgr.PerfCounter){

    if($pC.Level -ne 99){

      if(!$pctable.containskey($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType)){

        $pctable.Add(($pC.GroupInfo.Key + "." + $pC.NameInfo.Key + "." + $pC.RollupType),$pC.Key)

        $keyTable.Add($pC.Key, $pC)

      }

    }

  }

  # Test for a valid $Interval

  if($Interval.ToString().Split(" ").count -gt 1){

    Throw "Only 1 interval allowed."

  }

  $intervalTab = @{"RT"=$null;"HI1"=0;"HI2"=1;"HI3"=2;"HI4"=3}

  $dsValidIntervals = "HI2","HI3","HI4"

  $intervalIndex = $intervalTab[$Interval]

  if($EntityType -ne "datastore"){

    if($Interval -eq "RT"){

      $numinterval = 20

    }

    else{

      $numinterval = $perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod

    }

  }

  else{

    if($dsValidIntervals -contains $Interval){

      $numinterval = $null

      if(!$Start){

        $Start = (Get-Date).AddSeconds($perfMgr.HistoricalInterval[$intervalIndex].SamplingPeriod - $perfMgr.HistoricalInterval[$intervalIndex].Length)

      }

      if(!$Finish){

        $Finish = Get-Date

      }

    }

    else{

      Throw "-Interval parameter $Interval is invalid for datastore metrics."

    }

  }

  # Test if QueryMetrics is given

  if($QueryMetrics){

    $metrics = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

    $metricslist = @()

    foreach($pmId in $metrics){

      $pC = $keyTable[$pmId.CounterId]

      $metricslist += New-Object PSObject -Property @{

        Group = $pC.GroupInfo.Key

        Name = $pC.NameInfo.Key

        Rollup = $pC.RollupType

        Id = $pC.Key

        Level = $pC.Level

        Type = $pC.StatsType

        Unit = $pC.UnitInfo.Key

      }

    }

    return ($metricslist | Sort-Object -unique -property Group,Name,Rollup)

  }

  # Test if start is valid

  if($Start -ne $null -and $Start -ne ""){

    if($Start.gettype().name -ne "DateTime") {

      Throw "-Start parameter should be a DateTime value"

    }

  }

  # Test if finish is valid

  if($Finish -ne $null -and $Finish -ne ""){

    if($Finish.gettype().name -ne "DateTime") {

      Throw "-Start parameter should be a DateTime value"

    }

  }

  # Test start-finish interval

  if($Start -ne $null -and $Finish -ne $null -and $Start -ge $Finish){

    Throw "-Start time should be 'older' than -Finish time."

  }

  # Test if stat is valid

  $unitarray = @()

  $InstancesList = @()

  foreach($st in $Stat){

    if($pcTable[$st] -eq $null){

      Throw "-Stat parameter $st is invalid."

    }

    $pcInfo = $perfMgr.QueryPerfCounter($pcTable[$st])

    $unitarray += $pcInfo[0].UnitInfo.Key

    $metricId = $perfMgr.QueryAvailablePerfMetric($Entity.MoRef,$null,$null,$numinterval)

    # Test if QueryInstances in given

    if($QueryInstances){

      $mKey = $pcTable[$st]

      foreach($metric in $metricId){

        if($metric.CounterId -eq $mKey){

          $InstancesList += New-Object PSObject -Property @{

            Stat = $st

            Instance = $metric.Instance

          }

        }

      }

    }

    else{

      # Test if instance is valid

      $found = $false

      $validInstances = @()

      foreach($metric in $metricId){

        if($metric.CounterId -eq $pcTable[$st]){

          if($metric.Instance -eq "") {$cInstance = '""'} else {$cInstance = $metric.Instance}

          $validInstances += $cInstance

          if($Instance -eq $metric.Instance){$found = $true}

        }

      }

      if(!$found){

        Throw "-Instance parameter invalid for requested stat: $st.`nValid values are: $validInstances"

      }

    }

  }

  if($QueryInstances){

    return $InstancesList

  }

  $PQSpec = New-Object VMware.Vim.PerfQuerySpec

  $PQSpec.entity = $Entity.MoRef

  $PQSpec.Format = "normal"

  $PQSpec.IntervalId = $numinterval

  $PQSpec.MetricId = @()

  foreach($st in $Stat){

    $PMId = New-Object VMware.Vim.PerfMetricId

    $PMId.counterId = $pcTable[$st]

    if($Instance -ne $null){

      $PMId.instance = $Instance

    }

    $PQSpec.MetricId += $PMId

  }

  $PQSpec.StartTime = $Start

  $PQSpec.EndTime = $Finish

  if($MaxSamples -eq 0 -or $numinterval -eq 20){

    $PQSpec.maxSample = $null

  }

  else{

    $PQSpec.MaxSample = $MaxSamples

  }

  $Stats = $perfMgr.QueryPerf($PQSpec)

  # No data available

  if($Stats[0].Value -eq $null) {return $null}

  # Extract data to custom object and return as array

  $data = @()

  for($i = 0; $i -lt $Stats[0].SampleInfo.Count; $i ++ ){

    for($j = 0; $j -lt $Stat.Count; $j ++ ){

      $data += New-Object PSObject -Property @{

        CounterId = $Stats[0].Value[$j].Id.CounterId

        CounterName = $Stat[$j]

        Instance = $Stats[0].Value[$j].Id.Instance

        Timestamp = $Stats[0].SampleInfo[$i].Timestamp

        Interval = $Stats[0].SampleInfo[$i].Interval

        Value = $Stats[0].Value[$j].Value[$i]

        Unit = $unitarray[$j]

        Entity = $Entity.Name

        EntityId = $Entity.MoRef.ToString()

      }

    }

  }

  if($MaxSamples -eq 0){

    $data | Sort-Object -Property Timestamp -Descending

  }

  else{

    $data | Sort-Object -Property Timestamp -Descending | select -First $MaxSamples

  }

}

$start = Get-Date -Day 27 -Month 11 -Year 2013

$end = get-date -Day 28 -Month 11 -Year 2013

$ds = (Get-Datastore vcloud01).Extensiondata

Get-Stat2 -entity $ds -interval "HI2" -QueryMetrics | ft -AutoSize  ## OK

Get-Stat2 -Entity $ds -Stat "datastore.read.average" -Start $start -Finish $end  ## This doesn't work! See error message

This error message appear because it expects an "Interval" switch...like "HI2"...but souldn't this be unnecessary as long as I speicfy from/to dates ?  Anyway, I've tried both ways, and adding the ""-Interval" doesn't give me any latency numbers... 😕

Error message:

-Interval parameter RT is invalid for datastore metrics.

At C:\users\bok\Documents\04. Kundecaser\Conceptos AS\Visma IT\VMware\get-stats2.ps1:101 char:7

+       Throw "-Interval parameter $Interval is invalid for datastore metrics."

+       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : OperationStopped: (-Interval param...astore metrics.:String) [], Run

   timeException

    + FullyQualifiedErrorId : -Interval parameter RT is invalid for datastore metrics.


Thanks for helping out...really appreciate it!

Bjørn-Ove Kiil
Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

The default interval is the RealTime interval (RT), but the datastore metrics do not exist in that interval.

Add the Interval parameter.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos