How to fetch historical performance of ESx Server using vmware vi sdk

[vmcts]

Hi,

Can anyone let me know how can i retrieve the histrical performnace statistics such as CPU, Disk, Network etc of ESX Server. I am able to get the real-time

statistics data of cpu with the following code. I got to know from somewhere we can ignore the maxsample to get the historical statistics , is it true?? or do we need to make any changes in the code ??? Please reply.

// Get Performance Manager

ManagedObjectReference pmRef = _ServiceContent.perfManager;

// Get supported perf counters in the system

PerfCounterInfo[] cInfo = (PerfCounterInfo[])getObjectProperty(pmRef, "perfCounter");

// PerfInterval[] cInfo = (PerfInterval[])getObjectProperty(pmRef, "historicalInterval");

ArrayList vmCpuCounters = new ArrayList();

int index = 0;

for (int i = 0; i < cInfo.Length; ++i)

{

if ("cpu".Equals(cInfo[i].nameInfo.key))

{

vmCpuCounters.Add(cInfo[i]);

index++;

}

}

Hashtable usageCounter = new Hashtable();

while (index > 0)

{

//int index = 0;

for (IEnumerator it = vmCpuCounters.GetEnumerator(); it.MoveNext(); )

{

PerfCounterInfo pcInfo = (PerfCounterInfo)it.Current;

Console.WriteLine(index + " - " + pcInfo.nameInfo.label);

PerfCounterInfo pcInfo1 = (PerfCounterInfo)vmCpuCounters[--index];

usageCounter.Add((pcInfo1.key), pcInfo1);

//index--;

}

}

ManagedObjectReference vmmor = null;

_svcRef = new ManagedObjectReference();

_svcRef.type = "ServiceInstance";

_svcRef.Value = "ServiceInstance";

DateTime curTime = VS.CurrentTime(_svcRef);//"2009-05-10T13:53:12.547162Z"

//DateTime beginTime = curTime.Subtract(new TimeSpan(7, 0, 0));

DateTime NewbeginTime = curTime;//.Add(new TimeSpan(5, 32, 0));

//DateTime endTime = NewbeginTime;

DateTime NewendTime = NewbeginTime.Add(new TimeSpan(0, 5, 0));

ArrayList morlist = GetDecendentMoRefs(null, entityType);

if (morlist != null && morlist.Count > 0)

{

for (int j = 0; j < morlist.Count; j++)

{

//vmmor = (ManagedObjectReference)((object[])morlist[j])[0];

vmmor = (ManagedObjectReference)((object[])morlist[j])[0];

ObjectContent[] vmname = GetObjectProperties(vmmor, new string[] { "name" });

PerfMetricId[] aMetrics = VS.QueryAvailablePerfMetric(pmRef, vmmor, NewbeginTime, true, NewendTime, true, 20, true);

//PerfMetricId[] aMetrics = VS.QueryAvailablePerfMetric(pmRef, vmmor, DateTime.MinValue, false, DateTime.MaxValue, false, 20, true);

ArrayList mMetrics = new ArrayList();

if (aMetrics != null)

{

for (int index1 = 0; index1 < aMetrics.Length; index1++)

{

if (usageCounter.ContainsKey(aMetrics[index1].counterId))

{

mMetrics.Add(aMetrics[index1]);

}

}

}

//PerfMetricId[] metricIds = new PerfMetricId[50];

for (int count = 0; count < mMetrics.Count; count++)

{

PerfMetricId[] metricIds = (new PerfMetricId[] { (PerfMetricId)mMetrics[count] });

PerfQuerySpec qSpec = new PerfQuerySpec();

qSpec.endTime = NewendTime;

qSpec.entity = vmmor;

qSpec.maxSample = 1; ( ignore maxsample parameter to get the historical data ???)

qSpec.maxSampleSpecified = true;

qSpec.metricId = metricIds;

qSpec.intervalId = 20;

qSpec.intervalIdSpecified = true;

qSpec.startTime = NewbeginTime;

PerfQuerySpec[] qSpecs = new PerfQuerySpec[] ;

//Boolean continueDataCol = true;

//while (continueDataCol)

//{

PerfEntityMetricBase[] pValues = VS.QueryPerf(pmRef, qSpecs);

if (pValues.Length == 0)

{

Console.WriteLine("Virtual Machine " + vmname[0].propSet[0].val + " is not running" );

}

else

{

for (int i = 0; i < pValues.Length; ++i)

{

PerfMetricSeries[] vals = ((PerfEntityMetric)pValues[i]).value;

PerfSampleInfo[] infos = ((PerfEntityMetric)pValues[i]).sampleInfo;

Console.WriteLine("Sample time range: " + infos[i].timestamp.TimeOfDay.ToString() + " read every" + infos[i].interval + " seconds");

//+ " - " + infos http://infos.Length-1.timestamp.TimeOfDay.ToString());

for (int vi = 0; vi < vals.Length; vi++)

{

PerfCounterInfo pci = (PerfCounterInfo)usageCounter[vals[http://vi].id.counterId|http://vi].id.counterId];

if (pci != null)

Console.WriteLine(pci.nameInfo.label + "-" + pci.unitInfo.summary);

if (vals[vi].GetType().Name.Equals("PerfMetricIntSeries"))

{

PerfMetricIntSeries val = (PerfMetricIntSeries)vals[vi];

long[] longs = val.value;

for (int k = 0; k < longs.Length; ++k)

{

Console.WriteLine("instance:" + val.id.instance + " value:" + longs[k] + " ");

}

Console.WriteLine();

}

}

[lamw]

To gather performance information you can take a look at queryPerf() or by looking into the performance manager as you have, here are few additional threads/documents that may help you out:

There is a sample script in the vCLI called viperformance.pl that you can reference for query perf data

You also can find more information in the API forums: and there is also a specific C# forum for sample code/scripts found in the new VMware Code Central at:

=========================================================================

William Lam

VMware vExpert 2009

VMware ESX/ESXi scripts and resources at:

VMware Code Central - Scripts/Sample code for Developers and Administrators

If you find this information useful, please award points for "correct" or "helpful".

[vmcts]

Hi lamw,

Today i was exploring on Data Object - Perfprovider Summary object and got to know about the Perfprovider as:

This data object type describes capabilities of a performance provider. A performance provider is a ManagedEntity that can supply real-time and historical statistics. The summary indicates if either real-time (current), archived (past), or both statistical types are supported. For providers with currentSupported set to true, clients can call queryStats with the interval set to the provider's refreshRate to retrieve real-time statistics. If summarySupported is true, clients can call queryStats with the interval set to one of the historical intervals configured in the system to retrieve historical statistics for the entity.

What i understood from the above explanation is :

currentSupported = true and interval-id =refreshRate(default value is20) will give u the real time statistics .

and summarySupported true and interval-id = 300 (historical intervals configured in the system ( checked in the Data Object Type: PerfInterval and found 300 the sampling period) will give you the historical statistics.

so the real time and historical statistics is not dependent on the start and end time rather it depends on the interval value. is it true ??? Please let me know if i am wrongly understood.

If the above explanation is right then how would i differentiate the real time statistics and historical statistics. Also i noticed whenever the interval value is set as 300 , the max-sample parameter is ignored and the data is considered as historical data. whereas the other case i was getting the required samples(e.g maxsamples=2) if the interval value is set to default value. Please give me detailed information on performance.

Thanks

Pardeep Singh

[ehayes]

I find c# cumbersome to do somethings like this and perfer to use the VI Toolkit and powershell. In fact you can use a runspace in C# to execute powershell from your managed code.

See these docs for configuring stats levels and rollups in VC...I'm guessing your using VC. But powershell can be point to a stand alone host also.

-

---

Quick Powershell example----

---

#Connect to Virtual Center

connect-VIServer "myvc.mydom.com"

#collect metric for single vm

Get-Stat -Entity "VirtualMAchine001" -start "05/28/2009 00:00" -Finish "05/28/2009 23:59" -Stat "cpu.usage.average"

#Disconnect Virtual Center

Disconnect-VIServer -Confirm:$false

-

---

Complex Dynamic Example----

---

#Example to collect 24HR of data for all vms

#Create a .Net Datatable to hold results

$TableName = "Stats"

$table = New-Object system.Data.DataTable "$Stat"

$table.TableName=$TableName

#Create Colums

$col1 = New-Object system.Data.DataColumn Entity,()

$col2 = New-Object system.Data.DataColumn Type,()

$col3 = New-Object system.Data.DataColumn StartTime,()

$col3 = New-Object system.Data.DataColumn FinishTime,()

$col4 = New-Object system.Data.DataColumn Metric,()

$col5 = New-Object system.Data.DataColumn AVG,()

$col6 = New-Object system.Data.DataColumn MAX,()

$col7 = New-Object system.Data.DataColumn MIN,()

#Add the Columns

$table.columns.add($col1)

$table.columns.add($col2)

$table.columns.add($col3)

$table.columns.add($col4)

$table.columns.add($col5)

$table.columns.add($col6)

$table.columns.add($col7)

$table.columns.add($col8)

#Array of Virtual Centers to talk to.

$virtualcenters = "vcenter001.mydomain.com","vcenter002.mydomain.com"

#Array of Metrics to collect

$metrics = "mem.granted.average","cpu.usage.average"

#Time range to collect historical metrics from

$starttime = "05/28/2009 00:00"

$finishtime = "05/28/2009 23:59"

foreach ($vc in $virtualcenters )

{

#Connect to Virtual Center

connect-VIServer $vc

1. I use entity because vwmare uses the term...in this case an entity= a virtual machine.

1. Change Get-VM to Get-VMHost or Get-Cluster ...and you can reuse code to collect at a differant level

$entities = Get-VM

foreach ($entity in $entities)

{

foreach ($metric in $metrics )
{
#Write something to screen this may take awhile
Write-Host "Collecting: " $entity " " $metric " " $starttime " " $finishtime
#Get the stat insert row into datatable in memory
Get-Stat -Entity $entity -start $starttime -Finish $finishtime -Stat $metric -intervalmins 5 -maxsamples 1000 |
where { $_.Value -ge 0 } | Measure-Object -Property value -Average -Maximum -Minimum | foreach-object
{
$row = $table.NewRow();
$row.Entity = $entity.ToString() ;
$row.Type = $type.ToString() ;
$row.StartTime = $starttime ;
$row.FinishTime = $finishtime ;
$row.Metric = $metric.ToString() ;
$row.AVG = $_.Average ;
$row.MAX = $_.Maximum;
$row.MIN = $_.Minimum ;
$table.Rows.Add($row)
}
}
}
#Disconnect Virtual Center
Disconnect-VIServer -Confirm:$false

}

#Write datatable to screen or you could do any thing with it since it's a .net object maybe insert into SQL

$table

-

---

I attached code file for readability..........Message was edited by: ehayes

[vmcts]

Hi,

I am working on Real time performance statistics. I used the following function to get all the metric-ids along with the performance counters depending on the beginTime and endTime .

PerfMetricId[] aMetrics = VS.QueryAvailablePerfMetric(pmRef, vmmor, beginTime, true, endTime, true, intervalid, true);

where begineTime and endTime corressponds to server time(i.e the time at which the sample data is collected).

The following function i used to get the value of given metric-id:

PerfEntityMetricBase[] pValues = VS.QueryPerf(pmRef, qSpecs);

Now when i am trying to print the cpu real time data using the follwing function:

PerfMetricSeries[] vals = ((PerfEntityMetric)pValues[i]).value;

PerfSampleInfo[] infos = ((PerfEntityMetric)pValues[i]).sampleInfo;

Console.WriteLine("Sample time range: " + infos[i].timestamp.TimeOfDay.ToString() + " read every" + infos[i].interval + " seconds");

The datetime information does not seems to be matching with server time. Its almost 5 to 6 hours ahead.Can you let me know how should i validate my real time data . Can you please tell me infos[i].timestamp.TimeOfDay will print which date and time. I suppose it should print the server time(time from vi sdk method ). If true, then y the time is different from the server time while printing the value???

Regards

Pardeep Singh

[Rajeev_S]

Hi,

I was trying the script written by ehayes. Script doesnt throw any output, either it doesnt give any output. Guess the .NET datatable is not getting displayed. Below is the output i get,

Collecting: INCH-BGD138 mem.usage.average 05/28/2009 00:00 05/28/2009 23:59

Collecting: INCH-BGD138 cpu.usage.average 05/28/2009 00:00 05/28/2009 23:59

Any help is appreciated.

Thanks

[ehayes]

Sorry guys been on vacation...add this.

#Wrties Table to the screen

$table

#Write Table to Files Save Table

$output = "c:\temp\export.csv"

$table.Rows | Export-Csv -path $output -noTypeInformation -force

Message was edited by: ehayes - Fix format