High CPU utilization after upgrade to ESX 4 Update 1

[shrex]

I have just upgraded 2 hosts (IBM HS21 Blades) from ESX 4 to ESX 4 Update 1 and now I'm experiencing extremely high CPU utilization. Previously the utilization on each host would be around 10% but now after the upgrade it is around 80% on each and now all of the VMs are running very slow as a result.

The only change that has been made is that I downgraded the local hard disk drives from mirrored 500GB 7,200RPM SAS drives to mirrored 73GB 10,000RPM SAS drives and performed a clean install of ESX 4U1 on each host. As the 14 VMs are located on a SAN disk array I thought this would only improve the performance of the hosts.

Has anyone else had a similar issue or a potential fix?

Cheers,

Josh.

[AndreTheGiant · Accepted solution]

Seems very strange.

Do you have the same number of VM? Have you upgrade the VMware Tools?

There are other changes?

Local disk is not so important for overall performance.

Andre

[shrex]

There are the same number of VMs but only some of them have the updated VMware tools. My main problem with running the install for the tools are that the machines are running so slowly it's taking forever to update them.

Otherwise no other changes have been made.

I did find somewhere a potential fix for ESX 3 and 3.5 was to remove and recreate the HA cluster may solve the issue but I've tried it and it hasn't changed anything.

[AndreTheGiant]

Can you check each core usage? All the VM have the same high usage?

Do you see something strange in top and esxtop command?

The ESX configuration (vSwith for example) is the same?

Andre

[shrex]

Each processor cores usage is the same high usage.

I haven't need to use esxtop or top before so I will have to get back to you on that.

I've gone over the network configuration a few times to confirm it is correct and the same both hosts.

Even though I have confirmed the network settings are correct could it be a network issue because I've notice a few strange things going on with my management network which the service console of the ESX servers are connected to e.g. some packet loss and the inability to connect to some network devices on that network but there are no connectivity issues with the ESX servers themselves.

[psreekanth75]

Hi Shrex,

There can be many reasons for low performance issues and you seems to be tried and verified all the possible reasons.

Please raise SR and upload vm-support logs. Support team will look into the problem and come back to you with root cause.

Thanks

Sreekanth

[shrex]

I upgraded all of the VMware tools last night and everything is back to normal sub 10% CPU utilization on each host.

I didn't expect such a problem going from VMware tools version 4 on 4u1 hosts but I guess that what you get when you assume.

Thanks for helping me Andre.

Cheers,

Josh.

[AndreTheGiant]

Thanks for helping me Andre.

You're welcome.

PS: http://communities.vmware.com/thread/243404

Andre