VMware Cloud Community
Raudi
Expert
Expert

High CPU after vCenter update 7.0b -> 7.0c

Hi,

i just noticed, that the CPU load from around 30% jumps to around 70%. And this for the complete time since i installed the update.

Even after a reboot i have the same problem.

I found two hints:

The vAPI Endpoint shows a warning:

Failed to connect to 1da6ff8a-0bfd-4605-b4cc-c18ba520e95b\com.vmware.vcenter.nsxd.vapi vAPI provider.

And when using "top" on the vCenter CLI i have one process with high CPU:

  PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND

22892 root      20   0  753.5m  85.6m  98.7   0.5  30:11.56 S /usr/lib/vmware-wcp/wcpsvc --port 8920 --logfile /var/log/vmware/wcp/wcpsvc.log --configfile /etc/vmware/wcp/wcpsvc.yaml --audit-+

When i'm searching google for that service i found "Workload Management" this is for "Kubernetis", this i havn't licensed and i havn't NSX-D installed.

I stopped the serivce  "Workload Control Plane" and the CPU load goes down.

Has somebody seen this behavior too?

Looks like a bug to me...

Kind regards

Stefan

29 Replies
Raudi
Expert
Expert

I made some additional research, because when the service is stopped some other tasks don't work, like putting a host in maintenance mode.

I noticed only one thing:

Since the update there are no new log entrys written to this file:

/var/log/vmware/wcp/nsxd.log

Currently i don't want to open a support case and waste one of our "5" free cases for this problem...

I have stored my last successful backup of the VCSA, to revert to 7.0b and skip 7.0c, but i will wait a few days, perhaps someone can help here...

Reply
0 Kudos
guyrleech
Virtuoso
Virtuoso

I'm seeing very high CPU now after patching yesterday to 7.0.0.10600 and am investigating but top is showing /usr/lib/vmware-wcp/wcpsvc as being consistently using 90%+ CPU.

-- If you found this or any other answer useful please consider the use of the Helpful or Correct buttons to award points.
Reply
0 Kudos
guyrleech
Virtuoso
Virtuoso

https://vdr.one/vmware-vcenter-7-0-0c-high-cpu-usage/

-- If you found this or any other answer useful please consider the use of the Helpful or Correct buttons to award points.
Reply
0 Kudos
guyrleech
Virtuoso
Virtuoso

But I still have a high CPU python process after stopping the Workload Control Plane service - https://twitter.com/guyrleech/status/1290320296625537024?s=20

I did put a proper internal cert on my vCenter server last week which the web interface and PowerCLI are ok with. This is from /var/log/vmware/pod/pod-console.log.* where there are in excess of 271K log files.

vmware cert twistd error.jpg


-- If you found this or any other answer useful please consider the use of the Helpful or Correct buttons to award points.
Reply
0 Kudos
Raudi
Expert
Expert

The bug has been reported and waiting on the fix.

Has anybody informations if VMware will provide a fix for this, or if we must wait for 7.0d?

Reply
0 Kudos
RaoulSchaffner
Contributor
Contributor

i'm also seeing this.

thanks for the report that a fix is being worked on.

r.

Reply
0 Kudos
lemaya
Contributor
Contributor

Same here ... waiting for workaround

(Not helping but at least you guys know you are not alone 🙂

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee

This is going to be fixed in upcoming 7.0.0d.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Raudi
Expert
Expert

@ChrisFD2: Will the EP1 avilable on the patches page in MyVMware or must i create a support ticket to get it?

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee

It will be available via the patch portal however I would encourage all with this issue to create an SR so that GSS can keep you up to date on development.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
Raudi
Expert
Expert

O.k. thank you, but i can't create a SR, because we are a enterprise partner and have only 5 free cases in one year, so i will keep this cases for more important problems...

Reply
0 Kudos
ChrisFD2
VMware Employee
VMware Employee

Fair enough, once it's released I will try and remember to post an update here, but also keep an eye on the patch portal. I can't see it being much longer now.

Regards,
Chris
VCIX-DCV 2023 | VCIX-NV 2023 | vExpert *** | CCNA R&S
Reply
0 Kudos
goatman64
Contributor
Contributor

Any update here?  I checked via the vcenter update function earlier today and nothing ... thx

Reply
0 Kudos
depping
Leadership
Leadership

just checked the bug, a hot patch is being worked on. these things take some time unfortunately.

fehret
Enthusiast
Enthusiast

Hi there,

From my end, I did a vCenter rollback to 7.0b (and I regretted not to have taken snapshots for once at that time - too much trust!)

The high CPU and/or the impossibility to use maintenance mode by disabling the service was not practicable.

Waiting the patch, like the others! Smiley Wink

Cheers

Reply
0 Kudos
Maurizio71
Contributor
Contributor

vCenter 7.0.0d is out, I hope it will solve this issue.

Reply
0 Kudos
x01004753
Contributor
Contributor

I can't find anything on VAMI and neither on the Product Patches portal. Where do you have that info? Can you share a link?

Reply
0 Kudos
Raudi
Expert
Expert

I'm searching too, but in the product download, product patches download page i don't find a version "d".

And the server management told me that thare is no new versin too. The update planner the same...

Reply
0 Kudos