article Gathering virtual machine IOPS statistics by datastore in VMware PowerCLI Documents

Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Fri, 10 Sep 2010 01:41:54 GMT

Our friendly storage admin sometimes calls me up or sends alert my way from the array when the virtual environment is generating too many IOPS on a particular parity group. From his side, this could be happening on any of a number of LUNs that have been presented to our hosts, and he has no way of telling me what virtual machines could be causing the trouble.

Through the PowerCLI, I’ve scripted a way to check performance across the environment for the heavy hitters and report that info back. The script takes in an array of datastores, a vCenter Server FQDN, and a number of statistical samples to grab. It tracks down the physical hosts of the virtual machines that reside on those datastores and reports back the IOPS done (read & write) for each of those VMs over the interval specified. The credentials it asks for are a local account on the ESX/ESXi hosts, NOT for vCenter.

(Originally posted on my blog: http://virtualcurtis.wordpress.com)

Re: Gathering virtual machine IOPS statistics by datastore

LucD — Fri, 10 Sep 2010 11:15:56 GMT

Nice script.

____________

Blog: LucD notes

Twitter: lucd22

Re: Gathering virtual machine IOPS statistics by datastore

Zsoldier — Mon, 13 Sep 2010 19:22:07 GMT

Indeed. Nice script. One caveat is that stat tracking must be @ level 3 in vCenter for the disk.numberWrite.summation and disk.numberRead.summation statistics to show up.

Chris Nakagaki (Zsoldier)

http://tech.zsoldier.com

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Mon, 13 Sep 2010 20:14:56 GMT

Thanks guys.

@Zsoldier: Yeah, that's why I decided to get the stats from the hosts themselves. Would be a bit more streamlined if stats were up to level 3 across the board and everything happened through vCenter, but one of our environments is still at level 2.

Re: Gathering virtual machine IOPS statistics by datastore

burdweiser — Wed, 29 Sep 2010 16:12:15 GMT

Is there a way to specify multiple datastores or multiple hosts?

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Wed, 29 Sep 2010 16:43:02 GMT

The array that you pass to the -datastores parameter can have multiple datastores in it. For example:

GatherIOPS.ps1 -server myvcenterserver.mydns.com -datastores @("DS01","DS03","DS25") -numsamples 90

Currently, the script will only work against one vCenter instance (but gathers data from all ESX hosts managed by that instance). If there is a need to hit multiple vCenter Servers I can modify it to take this in as an array as well.

Re: Gathering virtual machine IOPS statistics by datastore

godbucket — Tue, 19 Oct 2010 14:43:07 GMT

Great script! Thanks for this!

Question: is there any way to supress the certificate errors that I'm receiving? (Sample below)

Thanks in advance!!!!

WARNING: There were one or more problems with the server certificate:

The X509 chain could not be built up to the root certificate.

The certificate's CN name does not match the passed value.

Re: Gathering virtual machine IOPS statistics by datastore

godbucket — Tue, 19 Oct 2010 15:13:49 GMT

Sorry to be a nuisance, but also is there anyway to output the IOP's in whole numbers (ie. take off everything after the decimals?)

Thanks again!

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Thu, 21 Oct 2010 13:15:57 GMT

godbucket -

I've uploaded a new version of the script that ignores the cert alerts. Also, if you want rounded numbers the easiest way would be to replace the last line in the GetAvgStat function:

return $totalIOPS/$samples/20

with this:

return ($totalIOPS/$samples/20)

Re: Gathering virtual machine IOPS statistics by datastore

godbucket — Thu, 21 Oct 2010 13:44:01 GMT

Cool, thanks for supressing the cert alerts!

Now when I try the GetAvgStat replacement function you mentioned above, I get the following error:

The term 'int' is not recognized as the name of a cmdlet, function, script file

, or operable program. Check the spelling of the name, or if a path was include

d, verify that the path is correct and try again.

At C:\Users\Tsmith\Desktop\GatherIOPS.ps1:78 char:12

+ return int <<<< ($totalIOPS/$samples/20)

+ CategoryInfo : ObjectNotFound: (int:String) [], CommandNotFound

Exception

+ FullyQualifiedErrorId : CommandNotFoundException

Please advise me on what to do, and thank you so much for your help!!!

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Thu, 21 Oct 2010 13:51:32 GMT

Sorry, looks like I need to learn how to format for these discussion boards... there should be brackets "[ ]" around "int"

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Thu, 21 Oct 2010 13:57:44 GMT

I went ahead and added the rounding to version 3 of the script after I realized that I don't really care that much if a single VM is doing .3 read IOPS.

Re: Gathering virtual machine IOPS statistics by datastore

godbucket — Thu, 21 Oct 2010 15:08:16 GMT

lol yeah, 0.2227262738 IOPS is hard to explain in reports to management.

Thanks so much for your script and your help. I appreciate it big time!!!!

Re: Gathering virtual machine IOPS statistics by datastore

mry — Tue, 09 Nov 2010 12:12:00 GMT

Hi really nice script

Would be cool if you could add " | where {$_.PowerState -eq "PoweredOn"} " after $myVMs = Get-VM -Datastore $myDatastore -server $server

So we dont get errors on powered off vms.

By the way, is there any way to do the same with the datastores ?

Re: Gathering virtual machine IOPS statistics by datastore

mry — Tue, 09 Nov 2010 12:31:31 GMT

and if you want to get all the datastores you can use

GatherIOPS.ps1 -server myvcenterserver.mydns.com -datastores @(Get-Datastore) -numsamples 90

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Wed, 17 Nov 2010 14:46:53 GMT

Would be cool if you could add " | where {$_.PowerState -eq "PoweredOn"} "
after $myVMs = Get-VM -Datastore $myDatastore -server $server
So we dont get errors on powered off vms.

Added!

By the way, is there any way to do the same with the datastores ?

Not sure I know what you mean here - you just want datastores with no running VMs to be excluded from the results? If so, yeah, that should be easy enough.

Re: Gathering virtual machine IOPS statistics by datastore

mry — Mon, 22 Nov 2010 12:10:29 GMT

What i mean is; Is it possible to get a list of IOPS per datastore(over a given time) insted for per vm(as in the script)

Re: Gathering virtual machine IOPS statistics by datastore

gtlscot — Tue, 08 Nov 2011 14:43:40 GMT

Hi. I am total neebie to running scripts. When I try and run your script I get several errors. I do get prompted for the ESX host account and password but which host is it talking about. Our hosts have several different passwords. Should we be logging in as root? Can I select specific hosts? Here are the errors I receive

Connect-VIServer : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:44 char:17
+ Connect-VIServer <<<< $server -NotDefault -WarningAction SilentlyContinue |
Out-Null
Get-Datastore : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:86 char:37
+   $myDatastore = Get-Datastore -Name <<<< $datastore -server $server
Get-VM : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:87 char:29
+   $myVMs = Get-VM -Datastore <<<< $myDatastore -server $server | Where {$_.P
owerState -eq "PoweredOn"}
Connect-VIServer : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:58 char:26
+     connect-viserver -server <<<< $vmhost -credential $credentials -NotDefau
lt -WarningAction SilentlyContinue | Out-Null
Get-Datastore : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
Attempted to divide by zero.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Connect-VIServer : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:58 char:26
+     connect-viserver -server <<<< $vmhost -credential $credentials -NotDefau
lt -WarningAction SilentlyContinue | Out-Null
Get-Datastore : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : The argument cannot be null or empty.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
Attempted to divide by zero.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)

Re: Gathering virtual machine IOPS statistics by datastore

lockenkeyster — Wed, 09 Nov 2011 18:43:41 GMT

You don't have to be logged in as root (although that works if they share the same password). This should also work with AD credentials if your hosts are configured to accept that. If not, then you will need to create local accounts on each ESX/ESXi host for this purpose. I haven't played around with that setup, but these accounts would likely only need read-only rights on the hosts to get this working.

Re: Gathering virtual machine IOPS statistics by datastore

gtlscot — Thu, 10 Nov 2011 10:22:36 GMT

Hi, I have now got a bit furtther. I created new accounts on nearly all of our hosts and the script begins to run but still generates errors but the script continues to run. However when the scripts completes it provides the names of the guests located in the LUN specified but does not provide any IOPs output. Errors are below.

+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:50:58    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:50:58    Get-VM        Could not find VIServer with name
'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:50:58    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Method invocation failed because [System.String] doesn't contain a method named
'op_Division'.
At C:\scripts\GatherIOPS.ps1:94 char:50
+       $data."Interval (minutes)" = ($numsamples*20)/6 <<<< 0
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:50:59    Get-Datastore        Could not find VISe
rver with name 'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:50:59    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:50:59    Get-VM        Could not find VIServer with name
'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:50:59    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:50:59    Get-Datastore        Could not find VISe
rver with name 'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:50:59    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:50:59    Get-VM        Could not find VIServer with name
'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:50:59    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Method invocation failed because [System.String] doesn't contain a method named
'op_Division'.
At C:\scripts\GatherIOPS.ps1:94 char:50
+       $data."Interval (minutes)" = ($numsamples*20)/6 <<<< 0
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:51:00    Get-Datastore        Could not find VISe
rver with name 'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:51:00    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:51:00    Get-VM        Could not find VIServer with name
'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:51:00    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:51:00    Get-Datastore        Could not find VISe
rver with name 'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:51:00    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:51:00    Get-VM        Could not find VIServer with name
'sose0003v.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:51:00    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Method invocation failed because [System.String] doesn't contain a method named
'op_Division'.
At C:\scripts\GatherIOPS.ps1:94 char:50
+       $data."Interval (minutes)" = ($numsamples*20)/6 <<<< 0
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:51:01    Get-Datastore        Could not find VISe
rver with name 's0011dev.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:51:01    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:51:01    Get-VM        Could not find VIServer with name
's0011dev.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:51:01    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
You cannot call a method on a null-valued expression.
At C:\scripts\GatherIOPS.ps1:68 char:28
+         if ($stat.instance.Equals( <<<< $myDatastoreID)) {
The operation '[System.Int32] / [System.String]' is not defined.
At C:\scripts\GatherIOPS.ps1:78 char:27
+     return [int] ($totalIOPS/$ <<<< samples/20)
Connect-VIServer : A parameter cannot be found that matches parameter name 'War
ningAction'.
At C:\scripts\GatherIOPS.ps1:58 char:86
+     connect-viserver -server $vmhost -credential $credentials -NotDefault -Wa
rningAction <<<< SilentlyContinue | Out-Null
Get-Datastore : 10/11/2011 09:51:01    Get-Datastore        Could not find VISe
rver with name 's0011dev.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-Datastore : 10/11/2011 09:51:01    Get-Datastore        Could not find any
of the servers specified by name.
At C:\scripts\GatherIOPS.ps1:61 char:34
+     $myDatastoreID = ((Get-Datastore <<<< $ds -server $vmhost) | Get-View).i
nfo.vmfs.extent[0].diskname
Get-VM : 10/11/2011 09:51:01    Get-VM        Could not find VIServer with name
's0011dev.scotland.gov.uk'.
At C:\scripts\GatherIOPS.ps1:64 char:40
+     $rawVMStats = get-stat -entity    (get-vm <<<< $vm -server $vmhost) -sta
t $stat -maxSamples $samples
Get-VM : 10/11/2011 09:51:01    Get-VM        Could not find any of the servers
specified by name.
At C:\scripts\GatherIOPS.ps1:64 char:40