tonygent
Enthusiast
Enthusiast

Using PowerCLI to get VM Disk Stats

Jump to solution

Firstly - this is really a message to LucD (who I know from previous assistance tends to prowl this forumSmiley Happy )  , just figured I'd share it, as it's all usefull info and any insight LucD has will no doubt be usefull for all.

Hi LucD,
Firstly - thanks for a great selection of posts on Reporting vSphere from Powershell - Well worth the time and a worth read for anyone.
I'm certainly not any good at PowerShell - but I was able to understand and make some changes for my benefit - so I figured I'd share my results and ask a couple of questions.
With your script as a basis - I've been able to come up with the following that SHOULD allow me to review ALL VMs in VC one at a time collect virtual disk stats for the VMs various disks, (IOPS, Throuput & Latency) I'm using VC 4.1 and and the storage is NFS attached, I believe these stats are OK but would appreciate your insight on this.
I'm quite pleased with the script - but it needs more testing on a valid site, rather than my small test area, but I'm sure from a 'PS' perspective there is more efficiency to be gained.
Firstly - is there any way of getting all the stats at once - rather than doing 6 calls per VM? (I'm also sure the '+=' one stat the the next can not be ideal)
Also - is there a way can control the output to the CSV, it seems to put colums in any order it likes and layout does not seem to be related to the group object command? Can this be controlled so I can have the 'value' and 'unit' entries next to each other?
Any insight and assistance would be appreciated, and I hope others will give it a test and let me know if it works for them too.
Thanks again for the inspiration and insight that got me this far Smiley Happy
Tony.
ESX 4.1, VC 4,1, PShell 2.0, PowerCLI 4.1.1
# Main Variables
$sVCentre = "VC IP"
$sVCUser = "administrator@domain"
$sVCPwd = "password"
$strVMWildCard = "*"
$strCSVName = "Stats-AvgVMDiskWriteStatsWorkingday"
$strCSVLocation = "c:\"
# define the start and finish times for a working day.
$today9am = (Get-Date -Hour 9 -Minute 0 -Second 0)
$today5pm = (Get-Date -Hour 17 -Minute 0 -Second 0)
$intStartDay = -1
$intEndDay = -1
## Begin Script
#Connect to VC
Connect-VIServer $sVCentre -User $sVCUser -Password $sVCPwd -ea silentlycontinue
$arrVMs = Get-VM | where-object {$_.Name -like $strVMWildCard}
foreach ($strVM in $arrVMs)
{
Write-Host "Getting Stats for : " $strVM.Name
Write-Host " - collating Write latency Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.totalWriteLatency.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host " - collating Read latency Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.totalReadLatency.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host " - collating Read IOPS Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.numberReadAveraged.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host " - collating Write IOPS Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.numberWriteAveraged.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host " - collating Read Throughput Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.read.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host " - collating Write Throughput Stats"
$stats += Get-Stat -Entity $strVM -Stat virtualDisk.write.average -Start $today9am.AddDays($intStartDay) -Finish $today5pm.AddDays($intEndDay)
Write-Host $stats.length
# group the data and collate the stats into averages for the day.
}$groups = $stats | Group-Object -Property {$_.Entity, $_.MetricId, $_.Instance}
$report = $groups | % {
New-Object PSObject -Property @{
Description = $_.Group[0].Description
Entity = $_.Group[0].Entity
EntityId = $_.Group[0].EntityId
Instance = $_.Group[0].Instance
MetricId = $_.Group[0].MetricId
Timestamp = $_.Group[0].Timestamp.Date.AddHours($_.Group[0].Timestamp.Hour)
Unit = $_.Group[0].Unit
Value = [math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 2)
}
}
#Exporting the report to a CSV file.
$strCSVSuffix = (get-date).toString('yyyyMMddhhmm')
$strCSVFile = $strCSVLocation + $strCSVName + "_" + $strCSVSuffix + ".csv"
$report | Export-Csv $strCSVfile -NoTypeInformation -UseCulture
61 Replies
LucD
Leadership
Leadership

This line

get-stat -Entity 'esxihost.fq.dn' | get-stats -Stat "datastore.numberReadAveraged.average"

should be something like this

Get-Stat -Entity (Get-VMHost -Name $strHostWildCard) -Stat $arrMetrics -Start $today9am.AddDays($intStartDay) -Finish $today5pm


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
_axel_
Contributor
Contributor

Tried that one as well, but no luck Smiley Sad .

What puzzles me is that we're using a small vmware cluster, 4 servers, one nfs-datastore for all of them, the setup isn't uniqe or special in any way, and still I can't get it to work.

I've attached the script that i'm using, and a screenshot from part of the output. I'm about to give up.

Thanks for all the help! I really appreciate it!

Best regards

/_Axel_

0 Kudos
LucD
Leadership
Leadership

Did you already try with my Get-Stat2 function ?

See Get NFS performance counters via PowerCLI


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
clavelstephane
Enthusiast
Enthusiast

Hello

Thanks a lot for this script.

I'm trying to find large vdisks with low iops in our infrastructure. With the script I can find the disks with low iops but I fail to modify it to add disk size (ideally used space).

How should I do this ?

thanks

0 Kudos
LucD
Leadership
Leadership

That is the 2nd script in this thread you are talking about, right ?

Note that it is not too obvious to link VMDK to guest OS partitions, or do you mean used space on the VMDK level ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

AlbertWT
Virtuoso
Virtuoso

Let us know here on in the new thread if there is any update LucD

/* Any kind of comment or input would be greatly appreciated */
0 Kudos
LucD
Leadership
Leadership

Not sure I understand what kind of update you are expecting ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

Hi,

I'm seeing some strange results with this and wondering if someone can help. If I run the script below, so without grouping, the results come out as expected, for both VMs.

$metrics = "cpu.usage.average","mem.usage.average","disk.usage.average","disk.maxTotalLatency.latest","net.usage.average"

$arrVMs = Get-VM Terminator, CCFS01

$start = (Get-Date).AddDays(-7)

$stats = Get-Stat -Entity $arrVMs -Stat $metrics -Start $start | fl

echo $stats

Value        : 0.58

Timestamp    : 02/03/2016 05:30:00

MetricId     : cpu.usage.average

Unit         : %

Description  : CPU usage as a percentage during the interval

Entity       : CCFS01

EntityId     : VirtualMachine-vm-107

IntervalSecs : 1800

Instance     :

Uid          : ...VirtualMachine-vm-107/FloatSample=cpu.usage.average\\635924934000000000/

Value        : 0.83

Timestamp    : 02/03/2016 05:00:00

MetricId     : cpu.usage.average

Unit         : %

Description  : CPU usage as a percentage during the interval

Entity       : CCFS01

EntityId     : VirtualMachine-vm-107

IntervalSecs : 1800

Instance     :

Uid          : ...VirtualMachine=VirtualMachine-vm-107/FloatSample=cpu.usage.average\\635924916000000000/

If I add a grouping.

$metrics = "cpu.usage.average","mem.usage.average","disk.usage.average","disk.maxTotalLatency.latest","net.usage.average"

$arrVMs = Get-VM Terminator, CCFS01

$start = (Get-Date).AddDays(-7)

$stats = Get-Stat -Entity $arrVMs -Stat $metrics -Start $start

$groups = $stats | Group-Object -Property {$_.Entity, $_.MetricId, $_.Instance}

$report = $groups | % {

    New-Object PSObject -Property @{

        Description = $_.Group[0].Description

        Entity = $_.Group[0].Entity

        MetricId = $_.Group[0].MetricId

        Timestamp = $_.Group[0].Timestamp

        Unit = $_.Group[0].Unit

        Value = [math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 2)

    }

}

echo $report

The output is only.

Value       : 52.34

Description : CPU usage as a percentage during the interval

Timestamp   : 08/03/2016 13:00:00

Entity      : CCFS01

MetricId    : cpu.usage.average

Unit        : %

Any help appreciated.

0 Kudos
LucD
Leadership
Leadership

I strongly suspect that the difference is in the use of the Instance property.

For a multi-core CPU there will be three different instances: the aggregate (instance -eq "") and one for each core (instance 0 and 1).

Same is true for the disk and net metrics (provided you have more than one of these in the VM)


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

The instance field was always empty. I tried removing the CPU metrics from the script and grouping only on Entity and MetricID, but still getting the same one line output.

$metrics = "mem.usage.average","disk.usage.average","disk.maxTotalLatency.latest","net.usage.average"

$arrVMs = Get-VM Terminator, CCFS01

$start = (Get-Date).AddDays(-7)

$stats = Get-Stat -Entity $arrVMs -Stat $metrics -Start $start

$groups = $stats | Group-Object -Property {$_.Entity, $_.MetricId}

$report = $groups | % {

    New-Object PSObject -Property @{

        Description = $_.Group[0].Description

        Entity = $_.Group[0].Entity

        MetricId = $_.Group[0].MetricId

        Timestamp = $_.Group[0].Timestamp

        Unit = $_.Group[0].Unit

        Value = [math]::Round(($_.Group | Measure-Object -Property Value -Average).Average, 2)

    }

}

echo $report

Value       : 65.11

Description : Memory usage as percentage of total configured or available memory

Timestamp   : 08/03/2016 13:30:00

Entity      : CCFS01

MetricId    : mem.usage.average

Unit        : %

EDIT: Even if I remove all metrics except mem.usage.average I still get the same result.

0 Kudos
LucD
Leadership
Leadership

How many elements do you get in the $groups variable after the Group-Object line ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

Just 1 server by the looks of it.

Count Name                  Group
----- ----                  -----

  668 CCFS01 mem.usage.average  {1.8, 1.75, 2.17, 2.06...}

0 Kudos
LucD
Leadership
Leadership

But since you group on EntityId and MetricId, there should be more entries.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

Yes exactly, something isn't quite right Smiley Happy

0 Kudos
LucD
Leadership
Leadership

Were these VMs poweredon at the start of he interval you are requesting ?

You can do a quick check in the Performance tab of the vSphere or Web Client.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

No, all powered on. Getting the same results regardless of which VM I pick. If you execute the scripts I posted what do you get?

0 Kudos
LucD
Leadership
Leadership

I get, as expected, 4 objects (one for each metric) per VM


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

Interesting.

Which vSphere/PowerCLI version? I'm using 6.0.

0 Kudos
LucD
Leadership
Leadership

The test was against vSphere 5.5, and I used PowerCLI 6.0


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

0 Kudos
warnox
Enthusiast
Enthusiast

Thanks for your help LucD.

I took a somewhat different approach but managed to make it work. I've put the solution on my site, http://docs.nzsystems.com/?p=262.

0 Kudos