vCenter 6.0 - high CPU usage!

[Sysxp]

Hello!

When we was on vCenter 5.5 its CPU consumption chart was nearly flat when idle and was about ~400-500 Mhz max.

Now we have this when upgraded to vCenter 6:

High peaks every 5 minutes - some "java.exe" process from SYSTEM account suddenly pops out, use several CPU cores and then disappears. It shows 3 times, and use 40-100 Mb each time.

After another 5 minutes it is repeated.

The host is ESXi 5.5 and the vCenter VM is the only VM on that host. vCenter is completly idle for hours - no vMotions, etc.

How to find out which vmware service exactly uses the CPU? Maybe it is possible to disable it, or somehow restrict it.

Thank you!

[kgruber]

Has anyone tried Update 3 yet? My VMware support tech says this fixes the problem. I would like to believe him but....... !

[sp4rt4n]

Hi,

same here on vCenter Server Appliance 6.0 U2.

I just want to share my experience with this...

Last year, I've deployed a vCenter Server Appliance 6.0 U2 (2 Hosts, 20VMs, 8 active), as you can see, CPU usage is spiking and "building up"

Yesterday, I've deployed a vCenter Server Appliance 6.0U3 (no Hosts/VMs), only time will tell how CPU will perform..

I've also deployed a vCenter Server Appliance 6.5U0 one month ago, here are the charts for comparison:

From my point of view: 6.5U0 is definitely better in performance comparison (Photon OS over SLES11 might be a reason for this)... however I would explicitly NOT recommend to use 6.5U0 as the vSphere Web Client is mandatory and ....a disastrous messy piece of administration software.

[abugeja]

Hey... Has anyone applied update 3 yet and has it resolve the high CPU issue?

[chicojr]

I too am having same issues, vcenter 5.5 on a win srv 2012 r2 box with 2cpu 16gb mem.

Vcenter hosts to datacenters with a total of 671 hosts and 1842 vms enabled.

Anyone have any idea what in the world is causing this intermittent spikes in cpu and mem?

[abugeja]

671 hosts to 1842 Vms - why are there so many hosts vs Virtual machines? I assume all the VM's are very large or you have very small hosts

[NTShad0w]

Hi,

as RaoulSchaffner observe on vCenter 6.0 on windows same situation is on vCSA Appliance 6.0 (U2 and U3 for now), and in 1 of my environment it consume 2200MHz of CPU and 80% of this is done by service(s):

vmware-vws

VMware System and Hardware Health Manager

so simple temporary workaround is to stop this service by typing (from vCSA shell):

service-control --stop vmware-vws

This service is responsible for HW monitoring, but as I know it's not working for vSphere C# Client in vSphere v6.0 so for smaller environments and admins that don't want to switch to terrible slow Web Client this is a choice in my opinion.

To not change anything in vCSA appliance it's safest to stops it after each vCSA 6.0 restart.

The results after applying this workaround is seen in attachment. Generally CPU usage drops from 2200MHz to acceptable 400MHz.

To VMware: Mates, please don't kill C# Client, it's your worst choice we notice from the beginning of your Company and great VMware VI3 and vSphere 4/5 product.

thanks

best regards

NTShad0w

[Sysxp]

I can't believe all this still happening in 2017! This is just... WOW. No words.

This is all went wrong since the WebClient introduction and the following deprecation of fast and beautiful C# client.

By the way, have you tried the all new HTML 5 "vSphere Client"? All I know is that it still can't do much.

[CollinChaffin]

Can't agree more.  In my almost 30yrs in this business VMware towers over any other vendor I've dealt with as far as totally and unequivocally ignoring the very paying customers that built and support their existence.

I have not, in real life with the Internet trolls aside, met a living human being ranging from those supporting large worldwide VMware infrastructures, all the way down to small single-host shops that do not share my (our) belief that this MONSTROSITY they call a "web client" is IN ANY WAY superior to the actual desktop application (C# client).  NOT ONE.

82 clicks and literally an HOUR to find just which magic "MANAGE TAB" (a term I now never want to hear again thanks to VMware) nested 17 screens/tabs deep, vs the THREE CLICKS in the C# client in a SINGLE WINDOW?  Hey VMware, COUNT ME IN!  Want to burn a HALF DAY just to figure out why a plugin that has ALWAYS appeared in the C# client suddenly disappeared in the magic web client?  ABSOLUTELY!  We in I.T. absolutely were not historically busy enough and really wanted our days stretched just a bit longer.  Thanks VMware!

IMO It's got to be bad inexperienced management listening to lazy inexperienced developers who didn't have the skills to code the C# client which probably started just after they lost enough dotnet DEV talent to continue the C# client support.  Just a theory but I cannot wrap my head around why else they would have continued down this path.

Unfortunately my friend, the ONLY way VMware will listen is if customers begin bailing with their $.  Personally, I have been with VMware as their top evangelist in every room since the first version of their workstation product and in the days of the old GSX server but I have recently begun to look at Hyper-V and other competitive products to replace VMware for this exact reason I refuse to fight to manage a product where the MFG with every revision seems to make my life HARDER, all while continuing to charge rising exorbitant licensing fees.

[CollinChaffin]

Oh forgot to post my screenshot to mirror what NTShad0w​ posted clearly showing my absurdly high CPU VCSA 6.0.0.30300 Build Number 7037817 before and after running:

service-control --stop vmware-vws

So I would say that NO, this has NOT been fixed in the latest VCSA 6.0u3 appliance patch available.

[Dave_the_Wave]

I also got the java.exe high cpu issue when I upgraded to VCS 6.0 u3c for Windows (2008 R2).

It was a patterned spike, exactly like all the graphs in this thread.

After the typical way to flip services on and off until you hit the culprit, it was the vmwarevws "VMware System and Hardware Health Manager" service that was causing this.

Stopping and setting the Windows Service to Manual puts the box back to sleep.

I felt the Health Manager was important, so I went back to VCS 6.0 u1b, and do not get the problem.

I still use the C# Client to do any cooking in the kitchen, but the VCS is nice to monitor and observe all is kosher.

[goppi]

I'm facing the same problem.

I setup a lab environment with ESXi 6.0U3 and VCSA 6.0U3f and CPU load is constantly between 20% and 50%.

This is a new Installation with one ESXi host and VCSA is the only VM running on that host.

Killing vmware-vws solves that problem but this is no solution.

Interestingly CPU load is quite OK after installation until the ESXi server is rebooted for the first time. I know this sounds strange but I verified it by completely starting from the very beginning and installed everything from scratch again. Same result.

I also verified that restarting VCSA does not cause the problem. It happened after first reboot of ESXI that causes this problem. From this point on nothing helped but deleting and installing VCSA from scratch again. But as already said this only lasts until ESXi is rebooted first time.

Update:

Just for completeness I installed latest VCSA 6.0U1 version, as from what I found in other posts problem begun when upgrading to 6.0U2 or later. My finding indeed support this assumption (see picture no 3). What is noticable is that the spikes on CPU load correlate with the messages in task pane "Acquire CIM ticket..." which you will find every 5 minutes for VCSA 6.0U1 but every minute on VCSA 6.0U1. Maybe this changed interval is the reason why CPU load is much higher on U3.

Before first ESXi reboot CPU is between 3% and 12%

After ESXi reboot CPU is between 20% and 50%

For VCSA 6.0U1 CPU is between 3% and 12% and a spike every 5 minutes

[DaneTruscott]

Did anyone get to the bottom of this.

I have just upgraded to V6U3h and the CPU load increased by 20% as describe by others on this post.

Stopping the "VMware System and Hardware Health Manger" Services Fixes the problem but its not really a fix as such.

Just want to Add,

No.1 is when i updated from 6.0.0, 3634793 (vCenter Server 6.0 Update 2) to 6.0.0, 9313458 (vCenter Server 6.0 U3h)

No2. This is when i stopped the "VMware System and Hardware Health Manager" Service

Thanks

[Dave_the_Wave]

I guess the bug is still there in U3h, but I didn't see anyone try U3g?

Again, the most recent version that does not have this issue is VCS 6.0 U1b, which I am using fine in a production environment.

[Sysxp]

Wow, this issue is here for some time now, yeah. All the way from 23.03.2015 to Dec 2018! Just ... wow.

My solution to this back then was simple - we rolledback and lived on 5.5 and it was great, primarily because of the C# vSphere Client.

Nowadays solution - we moved to ESXi 6.7 with VCSA 6.7.

Todays VCSA is a resource monster too - it always says "Need more memory" and its daily CPU chart looks like this:

In my opinion, however, it works more predictable and surely leaves a better feeling overall. (this does not include the installation process )

The only BIG dissapointment is the complete lack of the fast and bug free C# vSphere Client.

P.S. (why all of this happens)

- "Dad, what was the worst thing you know from your life?"

- "Three billion devices run Java."