VMware Cloud Community
bottomofbarrel
Contributor
Contributor

powercli script to capture cpu & mem usage stats

i am new to powercli and am not a good scriptor yet however, i need a script that will capture the max, min, avg cpu & mem usage stats per esxhost & vm for the past month and import into excel so i can create a pivot chart. key is i need to see the cpu/mem usage data per vm per host. can anyone help me with this. i have spent the last two weeks trying to work with powercli to do this and i am unable produce the results i want.

171 Replies
LucD
Leadership
Leadership

Not sure I get the question.

Are you talking about extracting the VMs that run a Linux OS, or do you mean specific Linux performance counters.

Because for vSphere there is no difference, the metrics Get-Stat returns are measured for the VM, independent of what OS is running in the VM.

If you want the same statistics for the Linux VMs, you'll have to find a unique string that is common in the OS Fullname to identify them.

Or you could just negate the Windows test

Get-VM | Where {$_.Guest.OSFullName -notmatch "Windows"}


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

YouyouL
Contributor
Contributor

That's why i am wondering why it's doesn't work (sorry i am a beginner in Powercli)  :

I got the error below when i am trying to check on "other than windows" OS VMs

Capture.JPG

Reply
0 Kudos
LucD
Leadership
Leadership

It could be that there are no metrics available for that counter for that VM for that time period.

The easiest way to check is to use the Performance tab in the vSphere Client for that VM.

Do you see data for that counter for that VM for that time period ?


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
YouyouL
Contributor
Contributor

You right !

Last question now, how can i convert this :

$vmstat.CPUMax = [int]$cpu.Maximum

  $vmstat.CPUAvg = [int]$cpu.Average

  $vmstat.CPUMin = [int]$cpu.Minimum

  $vmstat.MemMax = [int]$mem.Maximum

  $vmstat.MemAvg = [int]$mem.Average

  $vmstat.MemMin = [int]$mem.Minimum

Into percent ? It's really tricky i search an answer for that since 2 hours now Smiley Sad , in the perfect world, it should be cool to get something like that  :

VmNameTotal Mem in MBMemMaxMemAvgMemMinCPU in GHZCPUMaxCPUAvgCPUMin
VM1102470%30%2%3,345%10%3%

It's my goal actually

Reply
0 Kudos
LucD
Leadership
Leadership

Both counters, cpu.usage.average and mem.usage.average, are already expressed in percentage.

So there is no reason to convert.

Unless you mean something different.

If it is the percentage sign after the number, then you can

$number = 1.45

$percent = "{0:N}%" -f $number

$percent

or, if the number is already a percentage

$pnumber = 0.0145

$percent = "{0:P}" -f $pnumber

$percent

Note that $percent will be a [string] in both cases.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
YouyouL
Contributor
Contributor

Thanks a lot LucD for your precious help, i will buy your book to progress on Powercli !

Reply
0 Kudos
JohnMcKeown
Enthusiast
Enthusiast

MikeTrow wrote:

Great script that works fine in my small Vcenter however when I run it in a larger environment it stops part way through with the following error.

Get-Stat : 26/07/2012 11:00:21    Get-Stat        Object reference not set to a
n instance of an object.
At \\xxxxxxxxxxxxxxxxxxxxxxxx\Install\VMware\Migration_Tools\WIP\cpu-mem-stats4.ps
1:6 char:18
+ $stats = Get-Stat <<<<  -Entity $vms -Start $start -Stat $metrics
    + CategoryInfo          : NotSpecified: (:) [Get-Stat], VimException
    + FullyQualifiedErrorId : Core_BaseCmdlet_UnknownError,VMware.VimAutomatio
   n.ViCore.Cmdlets.Commands.GetViStats

Script adopted slightly to show vCPU and Memoray allocations attached.

The CSV is being created and populated but then stops.

Any thoughts ????

Connect-VIServer xxxxxxxxx

$allvms = @()

$vms = Get-Vm | where {$_.PowerState -eq "PoweredOn"}

$start = (Get-Date).AddDays(-30)

$metrics = "cpu.usage.average","mem.usage.average"

$stats = Get-Stat -Entity $vms -Start $start -Stat $metrics

$stats | Group-Object -Property {$_.Timestamp.Day},{$_.Entity.Name} | %{

$vmstat = "" | Select VmName, Day, MemAlloc, MemMax, MemAvg, MemMin, VCPU, CPUMax, CPUAvg, CPUMin

$vmstat.VmName = $_.Values[1]

$vmstat.Day = $_.Group[0].Timestamp.Date

$cpu = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property value -Average Maximum -Minimum

$mem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum

$vmstat.CPUMax = [int]$cpu.Maximum

$vmstat.CPUAvg = [int]$cpu.Average

$vmstat.CPUMin = [int]$cpu.Minimum

$vmstat.vCPU = $_.Group[0].Entity.NumCpu

$vmstat.MemMax = [int]$mem.Maximum

$vmstat.MemAvg = [int]$mem.Average

$vmstat.MemMin = [int]$mem.Minimum

$vmstat.MemAlloc = $_.Group[0].Entity.MemoryMB

$allvms += $vmstat

}

$allvms | Export-Csv "c:\VMs.csv" -noTypeInformation

Sorry to resurrect an old thread

The script is extremely useful, just there is a small error,

$cpu = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property value -Average Maximum -Minimum


Should read:


$cpu = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum


Does anybody have an idea of an easy way to add the VM resource pool information to it?


Thanks

John

Reply
0 Kudos
Konrailuck
Contributor
Contributor

Hi Everyone

          Please kindly help for check my script that i adapt from original post like below,

Connect-VIServer xxx -User yyy -Password zzz

$allvms = @()

$allhosts = @()

#$vms = Get-Vm | Where {$_.Guest.OSFullName -match "Windows"}

$vms = Get-Vm

foreach($vm in $vms){

  $vmstat = "" | Select VmName, CPUAvg, CPUMax, MemAvg, MemMax

  $vmstat.VmName = $vm.name

 

  $statcpu = Get-Stat -Entity ($vm) -Start "23/8/2014 06:00 AM" -Finish "23/8/2014 06:00 PM" -stat cpu.usage.average

  $statmem = Get-Stat -Entity ($vm) -Start "23/8/2014 06:00 AM" -Finish "23/8/2014 06:00 PM" -stat mem.usage.average

  $cpu = $statcpu | Measure-Object -Property value -Average -Maximum

  $mem = $statmem | Measure-Object -Property value -Average -Maximum

  $vmstat.CPUAvg = [math]::Round($cpu.Average,2)

  $vmstat.CPUMax = [math]::Round($cpu.Maximum,2)

  $vmstat.MemAvg = [math]::Round($mem.Average,2)

  $vmstat.MemMax = [math]::Round($mem.Maximum,2)

  $allvms += $vmstat

}

I just want,

1. Can i change time for run every 12 hrs ?

2. Can i get it to batch script file for run in schedule task?

3. If everyone want to suggest any idea from this, please ^^

Thank you.

Reply
0 Kudos
JohnMcKeown
Enthusiast
Enthusiast

Hi Guys

Just running the script from the outset again,

It seems my results are the same for each column and I am not getting the max and the mins, just the averages:

VmNameDayMemMaxMemAvgMemMinCPUMaxCPUAvgCPUMin
guest 18/23/2014 12:00:00 AM333111
guest 18/22/2014 12:00:00 AM333111
guest 18/21/2014 12:00:00 AM333111
guest 18/20/2014 12:00:00 AM444111
guest 18/19/2014 12:00:00 AM333111
guest 18/18/2014 12:00:00 AM141414333
guest 18/23/2014 12:00:00 AM333111
guest 18/22/2014 12:00:00 AM333111
guest 18/21/2014 12:00:00 AM333111
guest 18/20/2014 12:00:00 AM444111
guest 18/19/2014 12:00:00 AM777111
guest 18/18/2014 12:00:00 AM202020444
guest 18/23/2014 12:00:00 AM333111
guest 18/22/2014 12:00:00 AM333111
guest 18/21/2014 12:00:00 AM333111
guest 18/20/2014 12:00:00 AM444111
guest 18/19/2014 12:00:00 AM888222
guest 18/18/2014 12:00:00 AM191919555
guest 18/23/2014 12:00:00 AM000000
guest 18/22/2014 12:00:00 AM000000
guest 18/21/2014 12:00:00 AM000000
guest 18/20/2014 12:00:00 AM444111
guest 18/23/2014 12:00:00 AM666111
guest 18/22/2014 12:00:00 AM777111
guest 18/21/2014 12:00:00 AM666111
guest 18/20/2014 12:00:00 AM99922

2

Could this be because of the rollup jobs?

The version of the script I used is :

$allvms = @()

$vms = Get-Cluster 'MyCluster' | Get-Vm | where {$_.PowerState -eq "PoweredOn"}

$start = (Get-Date).AddDays(-30)

$metrics = "cpu.usage.average","mem.usage.average"

$stats = Get-Stat -Entity $vms -Start $start -Stat $metrics

$stats | Group-Object -Property {$_.Timestamp.Day},{$_.Entity.Name} | %{

$vmstat = "" | Select VmName, ResourcePool, Day, MemAlloc, MemMax, MemAvg, MemMin, VCPU, CPUMax, CPUAvg, CPUMin

$vmstat.VmName = $_.Values[1]

$vmstat.Day = $_.Group[0].Timestamp.Date

$cpu = $_.Group | where {$_.MetricId -eq "cpu.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum

$mem = $_.Group | where {$_.MetricId -eq "mem.usage.average"} | Measure-Object -Property value -Average -Maximum -Minimum

$vmstat.CPUMax = [int]$cpu.Maximum

$vmstat.CPUAvg = [int]$cpu.Average

$vmstat.CPUMin = [int]$cpu.Minimum

$vmstat.vCPU = $_.Group[0].Entity.NumCpu

$vmstat.MemMax = [int]$mem.Maximum

$vmstat.MemAvg = [int]$mem.Average

$vmstat.MemMin = [int]$mem.Minimum

$vmstat.MemAlloc = $_.Group[0].Entity.MemoryMB

$allvms += $vmstat

}

$allvms | Export-Csv "c:\VMsMonth.csv" -noTypeInformation

Reply
0 Kudos
snteran
Contributor
Contributor

I am very new to PowerCLI and powershell but have been tasked to collect some Memory and CPU stats to help get a hold of our available resources in our Cluster.  I tried the below script but it eventually just dies and the nothing is posted in the console window.  I then setup a Scheduled task but there is no CSV file that is created.  Any help / advise would be greatly appreciated:

#Add PowerCLI Snapin

Add-PSSnapin VMware.VimAutomation.Core

#Connect to cluster

Connect-VIServer vcenterServerName

$VmInfo = ForEach ($Datacenter in (Get-Datacenter | Sort-Object -Property Name)) {

  ForEach ($Cluster in ($Datacenter | Get-Cluster | Sort-Object -Property Name)) {

    $vms = Get-Cluster -Name $Cluster | Get-VM

    foreach($vmStat in (Get-Stat -Entity $vms -Start (Get-Date).AddMonths(-1) -Stat "mem.usage.average" |

    Group-Object -Property {$_.Entity.Name})){

      $memAvgGB = $vmStat.Group[0].Entity.MemoryGB * ($vmStat.Group | Measure-Object -Property Value -Average | Select -ExpandProperty Average) / 100

      ForEach ($HardDisk in (Get-HardDisk -VM $vmStat.Values[0] | Sort-Object -Property Name)) {

          "" | Select-Object -Property @{N="VM-name";E={$vmStat.Values[0]}},

              @{N="CPU";E={$vmStat.Group[0].Entity.NumCpu}},

              @{N="Uptime (Days)";E={(New-TimeSpan $vmStat.Group[0].Entity.ExtensionData.Summary.Runtime.BootTime (Get-Date)).days}},

              @{N="Memory Total (GB)";E={$vmStat.Group[0].Entity.MemoryGB}},

              @{N="Memory Avg (GB)";E={[Math]::Round($memAvgGB, 3)}},

              @{N="HDD";E={$HardDisk.Name}},

              @{N="HDD Capacity (GB)";E={$HardDisk.CapacityGB}},

              @{N="HDD Datastore";E={$HardDisk.FileName.Split("]")[1].Trim(' ')}}

       }

    }

  }

}

$VmInfo | Export-Csv c:\scripts\VMsInventory3.csv

Reply
0 Kudos
LucD
Leadership
Leadership

Can't you do a test run with a smaller set of your vSphere environment ?

Perhaps with 1 datacenter and 1 cluster.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
snteran
Contributor
Contributor

I'll give that a try, and report back.

I take it the implementation of the script is fine?

I had setup a batch file to call the ps1 file in the schedule task.  I have used this process for other powershell scripts I have utilized.

Thanks,

Reply
0 Kudos
LucD
Leadership
Leadership

The script runs without an issue in my test environment, but that is a rather limited environment.

Starting the script from a CMD file should cause no issues


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
snteran
Contributor
Contributor

Awesome, that worked.

I guess i'll have to work on breaking out the script so it can do one cluster at a time.  Is there a specific site you could recommend that would help explain the nuts and bolts of this script?  I'm going through the community and other blogs to learn some of the basics, just curious if you have some of your favorite sites.

Thanks again for your assistance,

Reply
0 Kudos
LucD
Leadership
Leadership

If you are particularly interested in vSphere statistics through PowerCLI, there are a number of posts on my blog.

Start with PowerCLI & vSphere statistics – Part 1 – The basics.

There is also a chapter on this in the PowerCLI Reference.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
Sivaramsharmar
Enthusiast
Enthusiast

Hi Jeveenj,

I have gone through your Script for getting report of VM Utilization stats for CPU & MEM.

I need one small help from you, I can able to see so many cmdlets like memmax, statcpu, etc., is there any complete referral for all these cmdlets in a single book which will help me to write scripts as per my requirement. Could you please direct me to that link.

Thanks in Advance.

Siva

Reply
0 Kudos
vThinkBeyondVM
VMware Employee
VMware Employee

Plz refer: vSphere 5.5 Documentation Center


----------------------------------------------------------------
Thanks & Regards
Vikas, VCP70, MCTS on AD, SCJP6.0, VCF, vSphere with Tanzu specialist.
https://vThinkBeyondVM.com/about
-----------------------------------------------------------------
Disclaimer: Any views or opinions expressed here are strictly my own. I am solely responsible for all content published here. Content published here is not read, reviewed or approved in advance by VMware and does not necessarily represent or reflect the views or opinions of VMware.

Reply
0 Kudos
marianofmaquiei
Contributor
Contributor

Hi guys,

I love the forum, great information! Thanks! , I can now retrieve all my statistics flawlessly!!

One small doubt though…

I use

$stat = Get-Stat -Entity ($cms)  -Realtime -stat mem.usage.average

And I get values like 9.99, 6.99, 7.99, 5.99 (I am expecting something around 80… as these are database vms where sql server is taking at least 80% of the vm memory…)

$statmem = Get-Stat -Entity ($cms)-start (get-date).AddDays(-7) -Finish (Get-Date)-MaxSamples 10000 -stat mem.usage.average

get values like 7.57, 8.26, 7.15, 9.42, 8.27 (same here; I am expecting something around 80… as these are database vms where sql is taking at least 80% of the vm memory…)

am I missing anything?

Thanks again!

Reply
0 Kudos
LucD
Leadership
Leadership

Note that the cpu.usage.average metric gives you the actively used virtual CPU over the total of available CPU.

For example, on an ESXi with a quadcore CPU, you have 1 x 4 x 2 = 8 pCPU (the last 2 comes from hyperthreading)

If you have a VM with 1 vCPU, that is used at 100%, it uses 1 pCPU completely, and the metric will show 1/8 = 12.5%.

The memory counter is the memory usage divided by the amount of configured memory (for a VM).

You find explanations of each counter under the PerformanceManager entry in the SDK Reference.


Blog: lucd.info  Twitter: @LucD22  Co-author PowerCLI Reference

Reply
0 Kudos
marianofmaquiei
Contributor
Contributor

Hi LucD,

Thank you very much for your prompt reply. I really appreciate it.

But I have this 2 cases, that I still don’t understand…

Server A: 32GB

Server B: 192GB

Server C: 140GB

Server A values for

Get-Stat -Entity ($servera) -Realtime -stat mem.usage.average

  1. 9.99, 6.99, 7.99, 5.99

Server B values for

Get-Stat -Entity ($serverb) -Realtime -stat mem.usage.average

  1. 10.99, 10.99, 13.99, 13.99, 12.99

Server C values for

Get-Stat -Entity ($serverc) -Realtime -stat mem.usage.average

  1. 0.99, 0, 0, 0, 0, 0, 0

You say: “The memory counter is the memory usage divided by the amount of configured memory (for a VM).”

Server A

Memory counter = mem usage/ config mem

  1. 9.99 = x / 32GB

Mem usage =  31968GB

Server B

Mem usage = 2110.08GB

Serve C

Mem usage = 138.6GB

Are my maths bad? It doesn’t make sense to me

Thanks again!!

Reply
0 Kudos