Hello. We have a vsphere 5.0 environment and we are experiencing some heavy io latency. I have been looking for a powercli script that will get the io latency for each vm and get the datastore name it currently resides on. We access our storage over fiber. I'm trying to get a good overview of io latency in one nice view in a csv. I found what might be a good base at https://communities.vmware.com/thread/304827?start=0&tstart=0 but I'm not quite sure how to get the datastore name into the array and I think it is written for nfs storage anyway. Thanks in advance for any info\advice!
This line
$start = (Get-Date).AddHours(-1)
into
$start = (Get-Date).AddDays(-20)
or
$start = (Get-Date).AddMonths(-1)
But remember that the interval time in these older metrics is increased due to the aggregation that is done.
See my PowerCLI & vSphere statistics – Part 1 – The basics post for some background info on this.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Thanks LucD, you're the man !
Hi LucD,
Script running successfully without Error.But no data in report.
i tried past two days and all . Still i didnt get any data in report
Is the statistical data available ?
Do you see the data under Performance in the vSphere client or the Web client ?
Could be that the Statistic Levels in your vCenter do retain these counters that long.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Hi LucD
Yes i can see
i am looking for last two days all VM's disk latency in data store wise in VC server
But i cant get
Can you attach the script you are currently using ?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
here the script
======================
$vmName = "*"
$stat = "datastore.totalReadLatency.average","datastore.totalWriteLatency.average",
"datastore.numberReadAveraged.average","datastore.numberWriteAveraged.average"
$entity = Get-VM -Name $vmName
$start = (Get-Date).AddHours(-1)
$dsTab = @{}
Get-Datastore | Where {$_.Type -eq "VMFS"} | %{
$key = $_.ExtensionData.Info.Vmfs.Uuid
if(!$dsTab.ContainsKey($key)){
$dsTab.Add($key,$_.Name)
}
else{
"Datastore $($_.Name) with UUID $key already in hash table"
}
}
&{Get-Stat -Entity $entity -Stat $stat -Start $start -ErrorAction SilentlyContinue |
Group-Object -Property {$_.Entity.Name} | %{
$vmName = $_.Values[0]
$VMReadLatency = $_.Group |
where {$_.MetricId -eq "datastore.totalReadLatency.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average
$VMWriteLatency = $_.Group |
where {$_.MetricId -eq "datastore.totalWriteLatency.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average
$VMReadIOPSAverage = $_.Group |
where {$_.MetricId -eq "datastore.numberReadAveraged.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average
$VMWriteIOPSAverage = $_.Group |
where {$_.MetricId -eq "datastore.numberWriteAveraged.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average
$_.Group | Group-Object -Property Instance | %{
New-Object PSObject -Property @{
VM = $vmName
Host = $_.Group[0].Entity.Host.Name
Datastore = $dsTab[$($_.Values[0])]
Start = $start
DSReadLatencyAvg = [math]::Round(($_.Group |
where {$_.MetricId -eq "datastore.totalReadLatency.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average),2)
DSWriteLatencyAvg = [math]::Round(($_.Group |
where {$_.MetricId -eq "datastore.totalWriteLatency.average"} |
Measure-Object -Property Value -Average |
Select -ExpandProperty Average),2)
VMReadLatencyAvg = [math]::Round($VMReadLatency,2)
VMWriteLatencyAvg = [math]::Round($VMWriteLatency,2)
VMReadIOPSAvg = [math]::Round($VMReadIOPSAverage,2)
VMWriteIOPSAvg = [math]::Round($VMWriteIOPSAverage,2)
}
}
}} | Export-Csv c:\report.csv -NoTypeInformation -UseCulture
=====================================================
Seems to be working for me without a problem.
Which PowerCLI build are you running ?
Do a
Get-PowerCLIVersion
And which vSphere version are you running this against ?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Power cli version 5.1 vsphere version 5.1
That should work with those versions.
Did you already try with just 1 VM ?
A VM for which you made sure that the data exists.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Yes I tried single VM also where it start $vmName = "*" And executed same script to different VC also. Script running successfully but no data
Then I'm out of ideas I'm afraid
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Dear LucD, I've tested the script more than 25 VC, Observed following points - Get the output with data exactly for Virtual Center Server 5.0 - Script running successfully but no data in output for Virtual Center Server 5.1 and appliance I'm not sure where should check, All VC setting are same 😞
Which of the properties in the report are incorrect on VC 5.1 ?
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
SIvapb,
Please try this.
Change the var start to this
$start = (Get-Date)
and then also change you get-stat statement to this.
&{Get-Stat -Entity $entity -Stat $stat -Realtime -ErrorAction SilentlyContinue |
When looking in Vsphere you may see data there, but its for realtime. In Vsphere if you click on the drop down only see realtime that is all you data for. So when you are looking for old data with this script it comes back with nothing. The changes I have stated will give you 1hour of data from the sec you run it.
Hi LucD,
Thanks for the script, I have used it and it works.
The result I'm getting is as follows for a VM.
Start : 2017-09-11 14:49:08
WriteLatencyAvg : 3276773906,3
Unit : millisecond
ReadLatencyAvg : 2149940644,19
VM : *************
Datastore : ******************
Is the read Latency Value is correct and the unit of value is ms?
The value is indeed in ms, but those values you are seeing look way off.
I did notice that sometimes there is an obviously incorrect value returned.
This might be caused by flaws in the statistical sampling process, since these values do look highly unlikely
I normally drop those extreme values.
In statistical circles this is called the "exclusion of outliers", but it is a subject on which opinions differ.
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Thanks for your reply..
I have used this script to find the VMs which is having high read latency to assign vFRC.
Unfortunately I don't have any clue now to identify VMs to assign vFRC.
Is there any script or any thoughts on how to identify the VMs for assigning vFRC?
I think the method you are using is the way to go (find the high latency vDisks), but throw away the extreme values, to get a more meaningful average.
Btw, this script looks at latency on the datastore level, you might perhaps also look at the vDisk latency values (see 3. Re: measuring_disk_latency_powercli)
Blog: lucd.info Twitter: @LucD22 Co-author PowerCLI Reference
Thanks Again LucD...
I have ran the script and it works.
In order to assign vFRC I have to check the latency when the workload runs on the server.
With this script (Re: measuring_disk_latency_powercli) I can get the Real Time value of latency in vDisk.
Is there a way to get historical data?