ESX network and disk I/O more than 100%. Why?

[eric_heilig]

I heard similar issues on VMs being attributed to shares configuration but these alarms are being generated on the ESX servers themselves. Please see the alarm output below.

If anyone can explain what would cause such high resource consumption, I am all ears. I cannot determine rational alerting thresholds on my environment without some means of tying these percentages back to my physical hardware.

thanks!

--Eric

Target: eg-farmzp-a001

Old Status: Green

New Status: Red

Current value:

ESX disk IO - (Metric Disk Usage (Average/Rate) = 355%)

Alarm: ESX disk IO

(\[Yellow Metric Is Above 80%; Red Metric Is Above 90%])

Description:

Alarm ESX disk IO on eg-farmzp-a001 changed from green to red

[Fabio_Pitzolu]

Hi

This disk is local or is it on a SAN? How many Vms are running on this VMFS? Do you have modified the shares values for those VMs?

Check these settings and post them, we'll see that.

Bye

Fabio

[ibrightly]

I also have similar alerts that occur on high disk usage. I'm running 20-30 VMs on each VMFS volume and they do not have modified shares values. I'm using an HDS 9585v SAN with SATA disk for these VMFS volumes.

Why would we ever get alerts that hit over 100%? Also what does 100% refer to? IO/s? If so, how many?

[eric_heilig]

My systems are running on EMC Symetrix SAN. I have approximately 40 VMs spread across 4 LUNs and all are running with default shares.

Eric

[eric_heilig]

I have confirmed with my VMware SE that this is not supposed to be possible and have opened a service request at his insistence. If anyone has any ideas then please chime in. Otherwise, I will post a resolution when one becomes available.

--Eric

[pauliew1978]

I have exactly the same problem. I set up the alarms and although they are triggered during times of high usage they state ridiculous amounts like 255 percent network usage etc. It would be real nice to sort this out so I can have some reliable alarms!!

[juchestyle]

What are the size of your LUNS?

Respectfully,

[ibrightly]

They vary a bit, but most of my LUNS on the HDS 9585v are 512GB.

[pauliew1978]

mine are 2tb

[juchestyle]

2 tb luns is huge, have you thought about scaling them back and possibly getting better performance?

Respectfully,

Matthew

[ibrightly]

We seem to not be alone with this problem. The documentation on disk usage seems quite non-existant. Check out this thread:

http://www.vmware.com/community/thread.jspa?messageID=729183

[pauliew1978]

well they are thin provisioned (they are actually 500gb) but serverd up to vmware as 2tb. We are using sanmelody.

[eric_heilig]

Luns are 500GB or less.

[eric_heilig]

All,

I have heard back on my service request. The responses were as follows:

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Hi Eric,

Looking further into these large percentages, it is apparently a bug in VC at this time. The numbers for disk IO and network are actually absolute (kb/s) and are not percentages. This will be addressed in an upcoming release/patch of VC.

Regards,

VMware Technical Support

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Eric,

It will most likely be addressed in the next release of VC which may be later this year. Unfortunately, I do not have a hard date, and I am unable to guarantee that it will come out by year's end.

Your case number has been added to the bug report for this issue. I will go ahead and close out this case unless you have any other concerns.

Regards,

VMware Technical Support

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

Just to confirm, we have already upgraded our environment to VC 2.0.2 and are still seeing the issues.

Good luck to all!

--Eric

[pauliew1978]

aahhh!!! absolute numbers rather than percentages. Now thats much better!!!!

Ok, I can live with that.

[ibrightly]

I agree. It'll be even better to have the 'bug' fixed in a future version of VirtualCenter.

[kpear]

My alarms are getting up to 1205% disk usage

Did this ever get fixed in VirtualCenter?

[ibrightly]

No, it does not appear to be fixed yet.