VMware Cloud Community
langleyj
Contributor
Contributor
Jump to solution

total consumption on cluster per vm

My environment consists of a virtual center with one cluster of 3 esx boxes. We also use VMotion to move around the 3 servers. Is it possible to get using powershell how much a single vm consumes of total resources on the vmware cluster and shoot that out into a csv? I ask for a single VM but really, I am looking in the csv to list all VM's and how much total resources (like mem.consumed.average

mem.sysUsage.average , and cpu.usage.average) they consume on the cluster.

Any help would be greatly appreciated!

Reply
0 Kudos
1 Solution

Accepted Solutions
LucD
Leadership
Leadership
Jump to solution

Let's tackle this step by step.

1) Statistical data for VMs that can have been vmotioned

There is a problem when you ask for -Realtime statistics.

The reason being that these data are kept on the ESX server, not on the VC.

The Get-Stat cmdlet only returns data for the ESX host on which the VM is currently running.

So if we want to see statistical data for VMs that have been vmotioned it is advised to go for one of the historical intervals !

Note that this is not a VITK problem, the QueryPerf method in the SDK displays the same behavior.

2) How to calculate the usage % against the cluster ?

I don't think the script should ask for percentages for the statistics (ex. cpu.usage.average).

In this case it's better to go for statistics that are expressed in a value for which we also have a total value on the cluster level.

What this means is the script should measure for example CPU usage in MHz (cpu.usagemhz.average).

3) What do we take as the available resources on the cluster ?

On a cluster we can do a straightforward summation of the available resources on each of the clusternodes.

For example, a cluster with 3 nodes where each node has a quadcore of 2.669 GHz would have 3 x 4 x 2.669 GHz of CPU resources available.

But is that what we want to measure the VM consumption against ?

On an ESX server (a clusternode) only part of the total available resources on the server is allocated to the guests.

Part of the available resources goes to the "system".

Taking the previous points into account the following script, in my opinion, expresses the CPU and memory usage of each VM as a percentage of the available cluster resources.

$report = @()

$cluster = Get-Cluster <clsutre-name> | Get-View
$clusterCPU = $cluster.Summary.EffectiveCpu
$clusterMem = $cluster.Summary.EffectiveMemory * 1Mb

$from = [Datetime]"03/05/2009 00:00"
$to = [Datetime]"03/05/2009 23:59"

Get-VM | % {
	$cpu = $_ | Get-Stat -Stat cpu.usagemhz.average -IntervalMins 5 -Start $from -Finish $to
	$mem = $_ | Get-Stat -Stat mem.consumed.average -IntervalMins 5 -Start $from -Finish $to
	for($i=0; $i -lt $cpu.Count; $i++){
		$row = "" | select VM, Timestamp, CPUperc, Memperc
		$row.VM = $cpu[$i].Entity.Name
		$row.Timestamp = $cpu[$i].Timestamp
		$row.CPUperc = "{0:N2}" -f ($cpu[$i].Value / $clusterCPU * 100)
		$row.Memperc = "{0:N4}" -f ($mem[$i].Value / $clusterMem * 100)
		$report += $row
	}
}
$report


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

View solution in original post

Reply
0 Kudos
28 Replies
LucD
Leadership
Leadership
Jump to solution

Let's tackle this step by step.

1) Statistical data for VMs that can have been vmotioned

There is a problem when you ask for -Realtime statistics.

The reason being that these data are kept on the ESX server, not on the VC.

The Get-Stat cmdlet only returns data for the ESX host on which the VM is currently running.

So if we want to see statistical data for VMs that have been vmotioned it is advised to go for one of the historical intervals !

Note that this is not a VITK problem, the QueryPerf method in the SDK displays the same behavior.

2) How to calculate the usage % against the cluster ?

I don't think the script should ask for percentages for the statistics (ex. cpu.usage.average).

In this case it's better to go for statistics that are expressed in a value for which we also have a total value on the cluster level.

What this means is the script should measure for example CPU usage in MHz (cpu.usagemhz.average).

3) What do we take as the available resources on the cluster ?

On a cluster we can do a straightforward summation of the available resources on each of the clusternodes.

For example, a cluster with 3 nodes where each node has a quadcore of 2.669 GHz would have 3 x 4 x 2.669 GHz of CPU resources available.

But is that what we want to measure the VM consumption against ?

On an ESX server (a clusternode) only part of the total available resources on the server is allocated to the guests.

Part of the available resources goes to the "system".

Taking the previous points into account the following script, in my opinion, expresses the CPU and memory usage of each VM as a percentage of the available cluster resources.

$report = @()

$cluster = Get-Cluster <clsutre-name> | Get-View
$clusterCPU = $cluster.Summary.EffectiveCpu
$clusterMem = $cluster.Summary.EffectiveMemory * 1Mb

$from = [Datetime]"03/05/2009 00:00"
$to = [Datetime]"03/05/2009 23:59"

Get-VM | % {
	$cpu = $_ | Get-Stat -Stat cpu.usagemhz.average -IntervalMins 5 -Start $from -Finish $to
	$mem = $_ | Get-Stat -Stat mem.consumed.average -IntervalMins 5 -Start $from -Finish $to
	for($i=0; $i -lt $cpu.Count; $i++){
		$row = "" | select VM, Timestamp, CPUperc, Memperc
		$row.VM = $cpu[$i].Entity.Name
		$row.Timestamp = $cpu[$i].Timestamp
		$row.CPUperc = "{0:N2}" -f ($cpu[$i].Value / $clusterCPU * 100)
		$row.Memperc = "{0:N4}" -f ($mem[$i].Value / $clusterMem * 100)
		$report += $row
	}
}
$report


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

Thanks LucD I think this may really be what the exec's are looking for Smiley Happy Simple questions...how can I use the Export-Csv cmdlet to export this out into csv format? I tried it..but I believe I put it in the wrong place.

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

Never mind, I got it!

$report | Export-Csv "C:\ClusterStats.csv" -noTypeInformation

Thanks LucD...this looks (I sure hope) like what they are looking for!

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

I was trying to understand this:

$cpu = $_ | Get-Stat -Stat cpu.usagemhz.average -IntervalMins 5 -Start $from -Finish $to

IntervalMins is over that time correct? I changed it to 15 from 5 and it still showed it as it was 5...am I misssing something?

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Yes, that's correct.

But you have to understand the principal of the Historical Intervals.

There are 4 of these and by default they run over 5, 30, 120 and 1440 minutes.

If you select a value in between the Get-Stat cmdlet calculates the nearest interval.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

Ahh thanks ...I did not know of that historical interval value. Curious..is it easy to chart this?

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

You can see these intervals in the VC Client.

In the VIC go to entry.

If by "chart" you mean at which -Intervalmins value Get-Stat would switch to another interval that should be easily automated.

Something like this perhaps:

$esx = Get-VMHost <ESX-server>
for($i=1; $i -lt 1450; $i++){
  $stats = $esx | Get-Stat -Stat cpu.usage.average -MaxSamples 2 -IntervalMins $i
  $delta = $stats[0].Timestamp - $stats[1].Timestamp
  Write-Host $i $delta.Minutes
}

You should see something like this

Note that for the 1 and 2 interval the statistical data interval is 20 seconds, which corresponds with the 5th interval, the Realtime interval.

1 0
2 0
3 5
4 5
5 5
6 5
7 5
8 5
9 5
10 5
11 5
12 5
13 5
14 5
15 5
16 5
17 5
18 30
19 30
20 30
21 30
22 30
23 30
24 30
25 30
....


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

I actually am curious if instead of exporting to excel, i can chart this using OWC or something. Since I need to show this to execs..pictures are always easier Smiley Wink

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

I see.

Actually I can imagine several ways of charting this.

1) automate (from with the PS script) the creation of graphs in Excel

2) use PowerGadgets, a PS snapin ($) that offers a number of good cmdlets for creating graphs

3) use PowerBoots, a library that allows a PS script to call all kind of graphical functions.

In fact I have been playing with PowerBoots recently and I'm quit impressed by this library.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

Brilliant! I will take a look at powerboots! Thanks!

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

LucD, I was going through this again and had a few simple questions...wonder what your thoughts are:

1. You used:

$clusterCPU = $cluster.Summary.EffectiveCpu

$clusterMem = $cluster.Summary.EffectiveMemory * 1Mb

Why the above over:

Summary.TotalCpu

Summary.TotalMemory

Side note, what is the logic behind the 1 mb?

2. You used:

Get-Stat -Stat mem.consumed.average

Why the above over:

mem.usage.average

Just trying to wrap my head around a few things here Smiley Happy

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

1. I used

$clusterCPU = $cluster.Summary.EffectiveCpu

$clusterMem = $cluster.Summary.EffectiveMemory * 1Mb

because those are the memory and cpu resources that are available to run virtual machines.

The properties totalCPU and totalMemory are all the memory and cpu resources that are available.

The difference between the two is that the Effective properties do not include the overhead from for example the console.

The division by 1Mb is to get the value in Gb.

See the ComputeResourceSummary object for more info on these properties.

2. In the Programming Guide, appendix A both metrics are described as follows:

  • usage : memory currently in use. This is active memory as a percentage of total available memory.

  • consumed : amount of host memory consumed by the virtual machine for guest memory

The primary reason for choosing "consumed" is that is expressed in Kb, while "usage" is expressed in percentage.

The secondary reason is that "usage" includes all memory used by the virtual machine while "consumed" expresses what is used for the virtual machine memory.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

langleyj
Contributor
Contributor
Jump to solution

Thanks! Once again you have cleared this up for me Smiley Happy

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

Is it possible to get DiskIO in this? I am not sure if that even makes sense across the cluster. I was also trying to see if I can get latency as well...I know I can do that through ESXTop...

Reply
0 Kudos
SinghSingh
Hot Shot
Hot Shot
Jump to solution

Luc, any chance you have a sample of the output from this script?

rajesh singh

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

Sure, this is from a small test cluster we use.

The name of the VM is erased.

I attached the file since the Insert Image button gives a system error


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
langleyj
Contributor
Contributor
Jump to solution

After some discussion with LucD, it seems that a bug was found in the original script. I am posting here the correct script which LucD has fixed as well as some details for anyone who may be interested:

clusterName = <ClusterName>
$report = @()

$cluster = Get-Cluster -Name $clusterName | Get-View
$clusterCPU = $cluster.Summary.EffectiveCpu
$clusterMem = $cluster.Summary.EffectiveMemory # In Mb

$from = [Datetime]"03/24/2009 00:00"
$to = [Datetime]"03/24/2009 23:59"

Get-Cluster -Name $clusterName | Get-VM | % {
# IntervalMins can be some other numbers as well...see earlier comments in this post
$cpu = $_ | Get-Stat -Stat cpu.usagemhz.average -IntervalMins 120 -Start $from -Finish $to  
$mem = $_ | Get-Stat -Stat mem.consumed.average -IntervalMins 120 -Start $from -Finish $to   # In Kb

for($i=0; $i -lt $mem.Count; $i++){
    $row = "" | select VM, Timestamp, CPUperc, Memperc
    $row.VM = $cpu[$i].Entity.Name
    $row.Timestamp = $cpu[$i].Timestamp
    $row.CPUperc = "{0:N2}" -f ($cpu[$i].Value / $clusterCPU * 100)
    $row.Memperc = "{0:N4}" -f ($mem[$i].Value / $clusterMem * 100 / 1Kb)
    $report += $row
    }
}

# I export to CSV but it can be graphed as well
$report | Export-Csv "D:\ClusterStats.csv" -noTypeInformation

Output of this would look something like:

VM Timestamp CPUperc Memperc

MyVM 3/24/2009 23:00 7.36 3.9338

MyVM 3/24/2009 21:00 6.25 3.4461

MyVM 3/24/2009 19:00 0.57 2.876

MyVM 3/24/2009 17:00 3.51 3.5603

MyVM 3/24/2009 15:00 7.4 3.8084

MyVM 3/24/2009 13:00 7.25 3.5009

MyVM 3/24/2009 11:00 0.6 2.8594

This can be read as 7.36% is the percentage CPU used by the guest calculated against the effective CPU and 3.9338% is the percentage Memory used by the guest against the effective Memory. Both of these values are across the entire cluster.

Reply
0 Kudos
JindiJee
Contributor
Contributor
Jump to solution

Guys, I've been trying to run this script with no luck. It writes out the CSV but with zero data. I am explicitly passing the cluster-name. I wonder if I need to be running VI Toolkit 1.5?

Also, i sthere a particualr reason why you don't have this script query all of the availa clusters as opposed to requiring a specific cluster?

Thanks,

Reply
0 Kudos
LucD
Leadership
Leadership
Jump to solution

The Get-Stat cmdlet had some problems in VITK v1 but has been much improved in VITK v1.5.

So yes, you better use VITK v1.5.

To run the script for all clusters in your environment requires a minor update.

This should do the trick.

$report = @()

$cluster = Get-Cluster -Name $clusterName | Get-View
$clusterCPU = $cluster.Summary.EffectiveCpu
$clusterMem = $cluster.Summary.EffectiveMemory # In Mb

$from = [Datetime]"03/24/2009 00:00"
$to = [Datetime]"03/24/2009 23:59"

Get-Cluster | % {
    $clusterName = $_.Name
	$_ | Get-VM | % {
		# IntervalMins can be some other numbers as well...see earlier comments in this post
		$cpu = $_ | Get-Stat -Stat cpu.usagemhz.average -IntervalMins 120 -Start $from -Finish $to 
		$mem = $_ | Get-Stat -Stat mem.consumed.average -IntervalMins 120 -Start $from -Finish $to # In Kb

		for($i = 0; $i -lt $mem.Count; $i++){
			$row = "" | select Cluster, VM, Timestamp, CPUperc, Memperc
			$row.Cluster = $clusterName
			$row.VM = $cpu[$i].Entity.Name
			$row.Timestamp = $cpu[$i].Timestamp
			$row.CPUperc = "{0:N2}" -f ($cpu[$i].Value / $clusterCPU * 100)
			$row.Memperc = "{0:N4}" -f ($mem[$i].Value / $clusterMem * 100 / 1Kb)
			$report += $row
		}
	}
}

# I export to CSV but it can be graphed as well
$report | Export-Csv "D:\ClusterStats.csv" -noTypeInformation


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos