network usage in vmware operations

[bjornb]

Hello,

In my environment vmware operations is complaining about 100 procent net i/o usage.

If i drill down further and do some math the 100 procent is impossible the virtual machine isnt even using 5 procent of the 1 gb.

also if i look in operations console the kbps isn't high.

I can't find anything how vmware operations does the 100 procent usage math.

i attached a screenshot of a virtual machine with only 10 kbps and already 12 procent net i/o ??

[admin]

Network usage numbers are not based on the local bandwidth, but the overall throughput possible based on observed values.   For example, you could have this gigabit network port that is attached to a gigabit switch passing traffic to a server over a 100kb WAN.  Because of this you cannot base your workload on the gigabit number, you need to estimate the true capacity based on usage.  We use a form of queuing theory to determine the capacity of Network and Disk i/o.

This metric should give users a fair sense of how much the network load is relative to the past usage. However, as noted, the numbers may be misleading in cases where there is a constant traffic at the same rate.

We are looking at some changes to how this metric is calculated in the next release of the product to make it more resilient.

[bjornb]

thanks for the answer, it makes more sense now.

but then again.

my virtual server is local on a local fysical switch on a 1 gb switch.

Looking at esx level and virtual machine level the network load is very low.

To often i have red bullets in vmware operations complaining about the net i/o

[admin]

Looking at your picture, the product is showing you are using 10,000  kbps -> 10 mbps which would be 1/10 of 100mbps.   So, it looks like the product is showing an effective throughput of ~100 mbps.   Its not entirely out of bounds.

[JaredBlack]

So how do you interpret 100% NET I/O on a host level?  If I'm reading this screenshot right, vCenter Operations believes that roughly 10 MBps is 100% on this host.  However, this host has all 1000/Full connections, and two load balancing on the vSwitch that handles VM network I/O (esxtop confirms that they're both being used as well).  Thanks.

[schistad]

I have to chime in on this thread as well.

We recently installed VCOPs in our environment and are having the same issue. Our infrastructure, ie core and ESX, is based on 10GbE yet VIOPs is flagging a load of, say, 6Mbps as 100% utilized and throwing up a red warning, both on the VM and the ESX host.

This clutters up the overview with false alarms and will quickly induce a tendency to ignore the Operations Manager status altogether since the normal state is to always have yellow and red alarms.

I am hoping that the autotuning will eventually eliminate most of these warnings, but I suspect that the way it works now it will always bring up a yellow or red alarm whenever a VM deviates from its norm. And this is not really desirable for this particular metric; Most systems have a very low network load in normal operations, but may have infrequent events - patching, file copying, etc - which do not follow any particular pattern, increase the throughput of the VM drastically, but are still completely safe and harmless events.

In fact, until a better algorithm to monitor network load is implemented it will probably be better - for us at least - if we were able to disable the network monitoring, as it is extremely unlikely that this particular metric will ever be a real issue for us.

[JaredBlack]

To be fair, in our case I don't doubt at all that it's accurately reporting network issues in our environment.  Our VMware environment is very dense, with the usual heavy network user suspects (antivirus, WSUS, SQL, etc.), and we have a lot of anecdotal evidence that suggests we need to upgrade to 10 GbE.  I would just like to know how vCenter Operations is determining when we hit the 100% threshhold for a host or individual VM on the network side, to better understand exactly what's going on and what's causing our current bottlenecks (ex: it could just be a few rogue VMs that need tuning).

[murphyslaw1978b]

I have the same issue.  I think it's because I'm on ESXi 4.1 update 0, and my NET traffic as reported in vCenter is always 0% (0 bytes).  I think they fixed this is 4.1 update 1, but I haven't upgraded yet.  Since vOPS pulls from vCenter, I think that's why the report is not accurrate.

[christg]

Hi, i just install the vcenter operation in evaluation mode , i have the same problem. many vm report NET I/O problem , when only use 5 -10 mps 

i notice this vm use flexible network adapter .

Network|Workload (%)						95
Network:Aggregate of all instances|Packets Received per second						8,004
Network:Aggregate of all instances|Usage Rate(KBps)						8,810
Network:Aggregate of all instances|Packets Transmitted per second						13,055