VMware Cloud Community
ejward
Expert
Expert

vCenter server at 100% CPU after upgrade to v4.1

vCenter server at 100% CPU after upgrade to v4.1. Memory if fine. It's virtual and Memory and CPU is fine on the host. It even runs at 100% when it's the only VM on the host. It's Server 2008 R2. Task manager show Virtual Center and Tomcat are taking the most CPU on the box. I'm not sure what else to look at.

Reply
0 Kudos
11 Replies
firestartah
Virtuoso
Virtuoso

do you have any reservations or limits setup? if so try remove/disable these.

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Gregg Robertson, VCP3,4 , MCSE, MCSA, MCTS, MCITP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Gregg http://thesaffageek.co.uk
Reply
0 Kudos
ejward
Expert
Expert

I have it in the highest resource pool and everything is ser to expandable and unlimited. I have no CPU reservation though.

Reply
0 Kudos
firestartah
Virtuoso
Virtuoso

when you say the highest resource pool, are you then stating you do have reservations?? Try remove the vm from the resource pool.

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Gregg Robertson, VCP3,4 , MCSE, MCSA, MCTS, MCITP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Gregg http://thesaffageek.co.uk
Reply
0 Kudos
ejward
Expert
Expert

No, there are no reservations. The only thing we are using resource pools for is to prioritize VMs in the event of resource contention. Since there's no way to tell if there has been contention, I don't know if recource poools are actually doing anything. I've asked Vmware during betas to maybe have alerts on resourse pools so we can see which two machines went after resources at the same time and which one got them based on the pools. This would help to adjust which machines are on which hosts and also let you see if resource pools are working. It would help you beeter make adjustments to pool resources rather than basically guessing.

Wow, did I get off track. We are just blindly using resource pools in the hopes that Vmware will make the right decision if there is contention. I know that was not the intention of resource pools but it seems that setting reservations and forcing limits could end up hurting you more than helping.

On another note, we do have a test resource pool with hard limits. Not in the same cluster as VC of course,

Reply
0 Kudos
firestartah
Virtuoso
Virtuoso

Have you tried reinstalling vmtools on the vc?

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful".

Gregg Robertson, VCP3,4 , MCSE, MCSA, MCTS, MCITP

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Gregg http://thesaffageek.co.uk
Reply
0 Kudos
ejward
Expert
Expert

Things have seemed to clam down by themselves. CPU is at about 58%. Of course, I have no baseline to compare this too so i don't know if this is unusual. All I know is that VC is now functional and that's great.

Because I have VC virtual, it's important to know what host it's on in the event at VC goes down. So I have DRS disabled for that VM. This way, it's always on the same host. Seemed like a good idea at the time but, could this be adding to my issues? When I do Intro to virtualization presentations I always talk about DRS and how, if a VM needs more resources, it moves to a host that has more resources. I know this is sort of backwards but, it's usually a non-technical crowd I'm addressing.

Could this be adding to the issue? I shut of DRS because I didn't want the VC server to move but, if the VC server spikes, does that mean that the other VMs won't move off the host?

Reply
0 Kudos
dgrace
Enthusiast
Enthusiast

Dsiabling DRS for the VM shouldn't be the issue. I'm running it that way for that same reason. I saw elsewhere that is is a common setup. Are you running the DB on the same server? Check disk utilization? How many vCPUs? Check HAL to make sure it is correct (uni vs smp)?

Reply
0 Kudos
firestartah
Virtuoso
Virtuoso

Hi

I am out of the office on holiday until the 20th of September.If you require development centre support please mail EMCCUKdevelopmentcentre@emc.com if you have not done so already

Regards

Gregg

If you found this or other information useful, please consider awarding points for "Correct" or "Helpful". Gregg http://thesaffageek.co.uk
Reply
0 Kudos
ejward
Expert
Expert

The CPU has gone down to 55% which, I still think is a little high. It has 2 CPUs and the correct HAL. The SQL 2005 DB is seperate.

Reply
0 Kudos
mark_chuman
Hot Shot
Hot Shot

Do you see any of these in the vpxd logs?

00112: 2010-08-20 22:17:21.591 06228 warning 'VpxdMoLock' **WARNING** Lock datacenter-26 mode EXCLUSIVE held for 1658 ms

Are your remote SQL servers clustered?

Do you see any vCenter, event id "1000"s in the windows application logs?

Any spike in %RDY for the vCenter VM?

Re: DRS - you should be able to set an affinity rule for the vCenter VM and still enable DRS, or leave DRS enabled and set the vCenter VM to manual or partial.

Reply
0 Kudos
ejward
Expert
Expert

Just an update.  This just sort of resolved itself.  Not sure what happened but, after a week or so, it calmed itself down.

Reply
0 Kudos