VMware Cloud Community
GordonPM
Enthusiast
Enthusiast

VM Stats report - inconsistent disk latency figures

I have the following script, which works for my purposes, except that the disk latency varies depending on how many days are being reported:

$metrics = "cpu.usage.average","cpu.usagemhz.average","mem.usage.average","mem.consumed.average","disk.usage.average","cpu.ready.summation","disk.maxtotallatency.latest"

$start = (Get-Date).AddDays(-2)

$dayStart = New-Object DateTime(1,1,1,6,30,0)     # 7:30 AM

$dayEnd = New-Object DateTime(1,1,1,8,30,0)      #  9:30 AM


$vms = Get-VM -Name 'servername'

$stats = Get-Stat -Entity $vms -start $start -stat $metrics


If($stats){

    $stats | Where-Object {

        $_.Timestamp.TimeOfDay -gt $dayStart.TimeOfDay -and

        $_.Timestamp.TimeOfDay -lt $dayEnd.TimeOfDay

    } | Group-Object -Property {Get-Date $_.Timestamp.Date -Format d} | ForEach-Object {

      $cpu = $_.Group | Where-Object {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum

      $cpuMhz = $_.Group | Where-Object {$_.MetricId -eq "cpu.usagemhz.average"} | Measure-Object -Property value -Average -Maximum -Minimum

      $mem = $_.Group | Where-Object {$_.MetricId -eq "mem.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum

      $memgb = $_.Group | Where-Object {$_.MetricId -eq "mem.consumed.average"} | Measure-Object -Property Value -Average -Maximum -Minimum

      $cpuRdy = $_.Group | Where-Object {$_.MetricId -eq "cpu.ready.summation"}| Measure-Object -Property value -Average -Maximum -Minimum

      $disk = $_.Group | Where-Object {$_.MetricId -eq "disk.maxtotallatency.latest"} | Measure-Object -Property Value -Average -Maximum -Minimum

      [pscustomobject]@{

            Day$_.Name

            CPUMaxMhz = [int]$cpuMhz.Maximum

            CPUAvgMhz = [int]$cpuMhz.Average

            CPUMaxPercent = [int]$cpu.Maximum

            CPUAvgPercent = [int]$cpu.Average

            CPURdyMaxPercent = [Math]::ROUND(($cpuRdy.Maximum/72000),2) #Calculation based on 2 hour sample size. Rounnd 2 decimal places.

            CPURdyAvgPercent = [Math]::ROUND(($cpuRdy.Average/72000),2)

            MemMaxPercent = [int]$mem.Maximum

            MemAvgPercent = [int]$mem.Average

            MemMinPercent = [int]$mem.Minimum

            MemMaxGB = [math]::Round(($memgb.Maximum/1MB),2)

            MemAvgGB = [math]::Round(($memgb.Average/1MB),2)

            MemMinGB = [math]::Round(($memgb.Minimum/1MB),2)

            DiskAvgLatencyMs = [int]$disk.Average

            DiskMaxLatencyMs = $disk.Maximum

            DiskMinLatencyMs = [int]$disk.Minimum

        }

    }

}

For some reason, when I run it with $start = (Get-Date).AddDays(-1) the disk latency figures look about right:

pastedImage_0.png


...but when I run it to get two or more days e.g $start = (Get-Date).AddDays(-2), for that first the figures are completely different

pastedImage_1.png


, which leads me to believe that none of the days are actually showing the info I want for disk latency in the required time period.

Tags (1)
Reply
0 Kudos
3 Replies
LucD
Leadership
Leadership

I suspect you might be seeing what aggregation does to your data.

The difference in the results is most probably due to the aggregation that took place between Historical Interval 1 and Historical Interval 2.

As an effect, your data that was averaged over a 5-minute interval (HI1) went to a data averaged over a 30-minute interval (H2).

Aggregation over longer intervals tends to flatten out the peaks and the valleys.

And I agree, become rather meaningless.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
GordonPM
Enthusiast
Enthusiast

That's... inconvenient to say the least as it's disk Max/Min latency I'm really interested in - backups are affecting VM performance at times of day and we're pretty sure it's disk latency....

Reply
0 Kudos
LucD
Leadership
Leadership

There is another aspect you have to consider and that is the statistics level you have defined for each Historical Interval.

When that is set to 1, you will have an average of all the maxima for all disks.
When set to 3, you get the maximum for each individual disk.

But higher statistical levels imply more space in the VCSA DB.

One solution I often used is to run a script at regular intervals and take the values for the Realtime interval.

Then store those values in an external file.

The drawback, you will have to have such a script running for some time before you have sufficient data to make conclusions.

Also important, at which interval do you run that script, every 5 minutes or every hour?

The smaller that interval, the more meaningful the data becomes of course.


This is somewhat emulating what vRA is doing.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos