VMware Cloud Community
maneeshlutz
Contributor
Contributor

vROPS and VM Guest shows CPU performance data mismatching

Hi All,

Recently during a performance test, we observed that the CPU performance data is mismatching between vROPS and VM guest OS. Environment details as below.

.

vROPS version 6.4

vCenter 6.0

ESXi 6.0

Host:     HP Proliant DL380 Gen9 Sockets: 2, Cores per socket:6, Logical proc: 24

VM:     2 virtual sockets, 6 Cores, total 12 vCPUs and 32 GB memory (Server 2012 r2 )

We are runing some applications on the VM and the CPU usage shows around 75 - 85 % where the vROPS metric 'CPU Usage%'  shows only 40 - 50%.

I understand that the vROPS is showing an average of 20 seconds. but in the guest, the CPU usage is consistent for at least 10 minutes.

Did anyone come across with the same observation, or can any one help in understanding that, what is wrong or what should be the metric which will give exact matching data as guest shows?

Thanks

Maneesh

Reply
0 Kudos
15 Replies
ryanrpatel
Enthusiast
Enthusiast

vROps data is typically an avg over a 30-day time period. Are you looking at the metrics of the Host or VM after waiting 15 to 20 min for collection? Does the VM have vmtools and is current? I haven't used vROps 6.4 in a long time since it's so old, but I can't imagine it's that far off the metrics. If you want real-time monitoring, I would use esxtop.

Reply
0 Kudos
maneeshlutz
Contributor
Contributor

ryanrpatel​ thanks for reply,

I am not looking for real time data. we may run performance often and collect data for the specific time frame for reference.

As I know, vROPS data is not an average of 1 month. It can provide data for shorter time intervals as well. also we can set the sampling intervals to minimum of 1 minute for a vCenter adapter.

VMware tools are running current version on the VM. I haven't observed esxtop while running performance. If so, can you suggest what counter should I look into, to get the VM CPU Usage %?

Reply
0 Kudos
GayathriS
Expert
Expert

This metric indicates the percentage of CPU that was used out of all the CPU that was allocated to the VM. CPU usage can indicate when the VM is undersized. Key: cpu|usage_average .

You have CPU USAGE for host and VM as well .

it depends on which you are looking.

regards

Gayathri

Reply
0 Kudos
maneeshlutz
Contributor
Contributor

GayathriS​ Thanks for reply

Yeah, I agree. In this instance I am looking for VM CPU usage.The dashboard is configured for  "cpu | usage_average"  of VM. I'm sure that the VM has sufficient resource . My concern is, why the vROPS data is mismatching with the data of guest.

Reply
0 Kudos
GayathriS
Expert
Expert

maneeshlutz​ Which Metric are you checking  on vROPS ?

regards

gayathri

Reply
0 Kudos
maneeshlutz
Contributor
Contributor

GayathriS

Below is how the first few lines of dashboard metric configuration looks like

<AdapterKind adapterKindKey="VMWARE">

        <ResourceKind resourceKindKey="VirtualMachine">

            <Metric attrkey="cpu|usage_average" label="" unit="" yellow="" orange="" red=""/>

Maneesh

Reply
0 Kudos
GayathriS
Expert
Expert

This Metric : cpu|usage_average  which is CPU|Usage (%) indicates the  percentage of CPU that was used out of all the CPU that was allocated to the VM.

CPU usage can indicate when the VM is undersized.

regards

Gayathri

Reply
0 Kudos
sxnxr
Commander
Commander

vrops collects every 5 mins and my understanding is that it is the average over that 5 min period.

Could not find the % but you will get the idea

CPU usages, as measured in megahertz, during the interval. 

VM - Amount of actively used virtual CPU. This is the host's view of the CPU usage, not the guest operating system view. 

Host - Sum of the actively used CPU of all powered on virtual machines on a host. The maximum possible value is the frequency of the two processors multiplied by the number of processors. For example, if you have a host with four 2 GHz CPUs running a virtual machine that is using 4000 MHz, the host is using two CPUs completely: 400 / (4 2000) = 0.50 Key: cpu|usagemhz_average

maneeshlutz
Contributor
Contributor

GayathriS

I understood that this is what that metric means. As I mentioned in the previous comment, I am wondering why it is not matching. ie. if the guest shows a value, vROPS also should show the same value or at least a value which is near to that of guest.

Reply
0 Kudos
sxnxr
Commander
Commander

In windows the cpu usage default view consolidates all physical and logical CPU’s into a single CPU view and is real time. You may see a content 70-80 % but that is the total real-time for all CPU added up into a single % so you could have 2 cpu that drop to 5% a lot and that would not show up in windows as it is a sum at the time converted to a %

Vrops looks at each vcpu usage value and averages it out over a 5 min period so the 2 that drop will effect the vrops calculations more heavily that what you see in windows

maneeshlutz
Contributor
Contributor

sxnxr​ Thanks for the detailed explanation

We have set the sampling interval as 1 min.

My understanding was, the CPU samples are average of every 20 sec. and  the 'cpu | usage_average' is the average  usage % of total CPU assigned to the specific VM. Though it is a host level view, it is not the average % CPU usage of total host CPU. Otherwise, there will be contribution from other VM running on the same host. The metric 'cpu | usagemhz_average', is of host. 

I will simplify my question. My whole intention is to figure out the CPU usage of an individual VM from vROPS and to compare the data with the CPU % measured from guest OS.

In detail, I have VM 1 and VM 2 running on a host. The CPU percentage measured in the guest level (Perfmon / Resource monitor) of VM1 is G%. Suppose the total CPU consumed on host level by both the VM is N%.

How can I figure out that, X% is used by VM1 and Y% is used by VM2?

Is X% is cpu | usage_average of VM1 and Y% is cpu | usage_average of VM2?

If yes, will G% and X% match, or at least will it show comparable data?

Reply
0 Kudos
maneeshlutz
Contributor
Contributor

sxnxr

In windows the cpu usage default view consolidates all physical and logical CPU’s into a single CPU view and is real time. You may see a content 70-80 % but that is the total real-time for all CPU added up into a single % so you could have 2 cpu that drop to 5% a lot and that would not show up in windows as it is a sum at the time converted to a %

Vrops looks at each vcpu usage value and averages it out over a 5 min period so the 2 that drop will effect the vrops calculations more heavily that what you see in windows

Does that mean, we cannot compare both guest and vROPS data (In this case of CPU usage %)?

or we should be able to match if we change the 'Overall Utilization' to 'Logical processors' view in the resource monitor window, which will show the individual CPU usage. (BTW, I haven't tried. probably need to try Smiley Happy )

Reply
0 Kudos
sxnxr
Commander
Commander

No what i am saying is you cant compare apples and oranges and wonder why they are different

Dont forget these metrics are based on two separate "objects" for lack of a better word

Perfmon is what windows sees as a sum of its processors(in this case they happen to be virtual)

vrops is an average of all core utilisation over a 5 min window on the physical cores in the host that the VM is using. That the vm is using is an impotent statement as well. It will also include any ESXi overhead the VM is generating outside of the VM.

They may never match. The only way to get a like for like in vrops for cpu currently is using the EPOS agent in the VM. VMware may add CPU to the list of in guest metrics that tools can pass but not added yet as far as i am aware.

What one you use i cant tell you that is up to you and what you want to know

maneeshlutz
Contributor
Contributor

sxnxr​ Thank you for nice explanation. It was helpful. Smiley Happy

Reply
0 Kudos
maneeshlutz
Contributor
Contributor

Thanks all for your response and time. Smiley Happy

Reply
0 Kudos