Performance data missing on one of three ESXi hosts

[FCBOE]

We are newbies only having our VMware installation since early May.  I noticed that one of our hosts and, subsequently, all the VM's on the host do not show performance data except in real time.  VCenter Server happens to be one of the VM's residing on this host.  I have read all the threads here about this issue, but have only found one that matches my dilemma.  This user suggested that all the VM's be migrated to other hosts, the host be removed from the cluster, the host be placed into maintenance mode, rebooted and moved back into the cluster.

Since I have never had to reboot any hosts, I am very nervous about trying to fix this issue.  I would appreciate any advice I can get.

[MauroBonder]

have two kb: follow

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=103081...

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=103608...

Please, don't forget the awarding points for "helpful" and/or "correct" answers. 

Mauro Bonder - Moderator

[FCBOE]

I don't believe either of the KB articles you listed is applicable to my situation since the Performance Graph data on my other two hosts is fine.  Please correct my assumption if I'm wrong.

[chriswahl]

I had this same problem on a host running ESXi 4.0u1. The host itself was not collecting performance data (as evident by collecting directly to it on the vsphere client), all was fine on the other hosts. The solution was upgrading to ESXi 4.1, which immediately began gathering performance data.

If you can connect to the host directly by vsphere client and see performance data, then I'd assume it's a vcenter issue.

I'd suggest using VMware update manager (it can be installed on the vcenter host) and scanning the host with the performance data issue, make sure it's up to date on all patches. The most current build of ESXi 4.1 that I'm aware of is 381591.

[FCBOE]

I do see performance data when I connect directly to the host with VCenter Client.  Also, since this is a new installation, we are already at ESXi 4.1.0 381591.  

[chriswahl]

One troubleshooting step down.

Make sure NTP is setup on that host, time sync is important in the environment for reporting performance.

Beyond that you could try a reboot of the host to see if it clears the issue up.

If that still doesn't work and you don't want to fiddle with vCenter, I'd raise a ticket with VMware support to dig deeper.

[FCBOE]

All three hosts point to the same NTP server.  I have never rebooted a host since our installation.  I read an article on rebooting the host and it doesn't seem too tough.  Would you consider this to be a serious problem that must be fixed or is it more of an annoyance?

[chriswahl]

I personally consider performance data important ... to troubleshoot issues and see how the health of the VMs are doing.

But more importantly, I consider rebooting a VMware host to be nearly zero risk. If you want to handle things manually, just start vmotion'ing off the guests to other hosts and make sure they are happy on their new hosts. Then put the "bad" host in maintenance mode (this just tells the cluster that the host should not be a candidate for any VMs and turns off HA on that host). Then reboot it.

That's the beauty of VMware, no outage windows for the host's guests.

[FCBOE]

The reason I am so vary of rebooting this host is that my trainer demoed HA by rebooting this particular host which threw an error.  I believe he had to turn HA back on and I'm not sure I can do that.  I think we had a similar problem when we patched the hosts - this particular one had an issue coming back up.  Since I am the sole support of this environment, I am hesitant to even try it.

[FCBOE]

I just found a KB article that suggests if only one host is failing to report performance data, then the Management Agents should be restarted on the host.  Have you ever performed this task?

[bulletprooffool]

Hi - restarting the management agents is zero risk - just go ahead and do it (in fact, you won't even see any impact on VMs)

It sounds though as if the relationship between your ESX host and VC is not quite what it should be - I'd recommend removing the host from your VC and re-adding it. In addition, I'd verify the DNS settings on your ESX host and your VC(I assume the VC is OK - as this problem is unique to 1 host)

[FCBOE]

I restarted the management agents, but the performance data problem remains.  Should I reboot the host?  Since my VCenter Server runs on that host, I assume I should definitely migrate it to another host.  Is that correct?

[FCBOE]

I opened a Service Request and was instructed to remove the host from Inventory and add it back in.  That worked - all the performance data was available.  However, I was missing the BIOS version and release date on the Configuration Tab and some of the software sensors on the Hardware Status tab.  A VMware tech restarted the Management Agent services and that fixed the BIOS and sensor problems.

[lclute]

We are having the same issue. What it sounds like is a problem with the Management Agents on the ESXi Host.  Wanted to find out if there is a patch for this issue or if there are any scripts available to check the status of ESXi Host Agents to identify those that are not functioning correctly.  Right now the only way I know is when I see that a Host is not showing up on my nightly collection of metrics.  Than I have to go into VC and check the performance tab of the server and see if it is collecting any historical data.  We have noticed that this issue just started happening when we went to ESXi 4.1.  Anyone else experiencing this issue as well?