VMware Cloud Community
sean0xb8
Contributor
Contributor

Hardware - Health Status Monitoring - CIM - alarm/error - "Cannot access the hardware monitoring service"

Problem:

Our vCenter reports back that it "Cannot access the hardware monitoring service" when we click on the Hardware Status tab.

Setup:

-- two vsphere hosts running ESXi 5.1.

-- vCenter on a third machine running Windows 2008R2

-- All three machines are on the same network and subnet



Verified:

-- from vCenter, I verified that CIM Server service is running by checking Configuration (Tab) -> Security Profile -> Services

-- also from vCenter, I checked firewall setting by looking Configuration (Tab) -> Security Profile -> Firewall; according to the Firewall page, the CIM Server service runs on both TCP ports 5988 and 5989 -- but this may not mean anything because vCenter source port might be different from 5988 or 5989


Troubleshooting attempts:

-- restarted CIM Server service



Known issues:

-- checked and determined that for both ESXi hosts, port 5989 is open but 5988 is closed

-- ran wireshark packet capture on vCenter - I checked and determined that no TCP packets target port 5989 nor 5988 on the ESXi hosts from vCenter server



Question:

I would appreciate help on how to proceed forward.

1. I figure I want to open up port 5988 on the ESXi hosts - how can I do this?

2. Is there a CIM Client on vCenter that I need to check - to see if it is running?



Thanks,

Sean


21 Replies
TBKing
Enthusiast
Enthusiast

There is a "vCenter Inventory Service" that runs on the vCenter server.

Ours wouldn't start until we put an exception on our Virus package.

sean0xb8
Contributor
Contributor

Thanks for the suggestion. I checked the vCenter Inventory Service through Windows Adminstrative Tools -> Services. I see that the Inventory Service is currently running with parameters set to Automatic and Log on as Local System.

In terms of the anti-virus software, the vCenter has AVG Internet Security Network Edition 9.0.930 running on it. I don't see anywhere to add exceptions to the scanner except for specific file extensions.

Checking the path of the executable associated with the Inventory Service points to a directory with multiple files - but I don't know which file is attached to the service because the label is cut off from "Properties"

Directory of C:\Program Files\VMware\Infrastructure\Inventory Service\bin

11/09/2012  07:03 PM    <DIR>          .

11/09/2012  07:03 PM    <DIR>          ..

10/19/2012  06:15 PM           161,280 jwinauth.dll

10/19/2012  06:15 PM         1,291,776 libeay32.dll

10/19/2012  06:15 PM           348,160 openssl.exe

10/19/2012  06:15 PM           261,632 ssleay32.dll

10/19/2012  06:15 PM           135,168 zip.exe

Reply
0 Kudos
sean0xb8
Contributor
Contributor

One thing I would like to know is how to determine or set which port - TCP:5988 or TCP:5989 - vCenter targets for Hardware Status monitoring.

Just to reiterate - only the 5989 port appears to be open on the ESXi hosts and wireshark capture from vCenter doesn't show any packets targeting either 5988 nor 5989.


I forgot to mention previously - it appears 5989 is the port for secure communication.

Reply
0 Kudos
TBKing
Enthusiast
Enthusiast

Ports:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=101238...

Virus / Inventory:

If it's running, it should be fine.  My Anti-Virus was not letting the service start at all because it was accessing a file with a flagged name.

From our AV guy:

File getting blocked: C:\Program Files\NetApp\Virtual Storage Console\lib\google-collect-1.0.jar

We block any files named “google*.dll”, “google*.jar” and “google*.exe”. 

Reply
0 Kudos
sean0xb8
Contributor
Contributor

I did some more digging and found additional information that supports the network setup where both the vCenter host and ESXi hosts must have TCP port 5989 open to other machines. What I am not clear on is if the ESXi host has TCP port 5988 only open internally to itself the localhost whereas vCenter its TCP:5988 open to other machines.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=203909...

http://kb.vmware.com/selfservice/documentLinkInt.do?micrositeID=&popup=true&languageId=&externalID=2...

Reply
0 Kudos
sean0xb8
Contributor
Contributor

from VMware vSphere 5.1 Documentation Center

vCenter 5.1 extension.PNG

The Hardware Status tab displays the following error message:

the remote name could not be resolved SERVER_NAME

where SERVER_NAME is the domain name of the vCenter Server system.

This error appears when the client system is unable to resolve the domain name of the vCenter Server system. Either fix the domain name resolution problem, or edit the file \extensions.xml on the vCenter Server system and replace the vCenter Server domain name with its IP address.

Would like to compare the four below with other group's setup

In my group's setup -

1. file is called C:\Program Files\VMware\Infrastructure\VirtualCenter Server\extensions\cim-ui\extension.xml -- notice it is singular and not plural

2. the file itself (extension.xml) references scriptConfig.xml, but there is not file scriptConfig.xml in our directory (C:\Program Files\VMware\Infrastructure\VirtualCenter Server\extensions\cim-ui)

3. there is no domain name or ip reference to the vCenter server in the extension.xml file, but rather it states - "<url>https://*:8443/cim-ui/scriptConfig.xml</url>"

4. is it suppose to be 8443 or 5989 in this file?

dir output

C:\Program Files\VMware\Infrastructure\VirtualCenter Server\extensions\cim-ui>dir

Directory of C:\Program Files\VMware\Infrastructure\VirtualCenter Server\extens

ions\cim-ui

10/07/2013  11:35 AM    <DIR>          .

10/07/2013  11:35 AM    <DIR>          ..

11/12/2012  11:44 AM               790 extension.xml

11/12/2012  11:44 AM    <DIR>          locale

               2 File(s)          1,580 bytes

               3 Dir(s)   1,107,329,024 bytes free

contents of extension.xml

<?xml version="1.0" encoding="UTF-8"?>

<config>

   <extension>

      <name>cim-ui</name>

      <version>5.1</version>

      <company>VMware, Inc.</company>

      <description>

         <label>vCenter Hardware Status</label>

         <summary>Displays the hardware status of hosts (CIM monitoring)</summary>

      </description>

      <certificate>ssl/rui.crt</certificate>

      <keyFile>ssl/rui.key</keyFile>

      <servers>

         <server>

       <url>https://*:8443/cim-ui/scriptConfig.xml</url>

       <description>vCenter Hardware Status</description>

       <company>VMware, Inc.</company>

       <type>com.vmware.vim.viClientScripts</type>

       <adminEmail>noreply@vmware.com</adminEmail>

         </server>

      </servers>

   </extension>

</config>

Reply
0 Kudos
ramkrishna1
Enthusiast
Enthusiast

Hi

Welcome to the communities.

As you have already did lot of exercise so nothing need to work except hotfix & HCL product.

  

"I take a decision and make it right."
Reply
0 Kudos
TBKing
Enthusiast
Enthusiast

Same here - "extension.xml" (not plural)

File looks exactly the same as yours except I'm at 5.0

The scriptConfig.xml is under

Program Files\VMware\Infrastructure\tomcat\webapps\cim-ui.

Open a browser to "https:\\server.name:8443"

and you should get the Tomcat default page.

... at least this is how I tracked down the scriptConfig.xml.

That file has nothing special in it either.


Reply
0 Kudos
sean0xb8
Contributor
Contributor

Thanks for the reply. I did not completely understand your suggestion - are you saying that I need to verify my hardware compatibility?

My group uses two rackmounts - HP ProLiant DL320 G6 with Xeon CPU E5504.

It is listed in the VMware Hardware Compatibility page. I didn't see anything in the known issues section corresponding to hardware monitoring.

Additionally, I verified that port 8443 is open and listening on the vCenter Server by typing in a web browser on a client machine -- https://<vCenter Server IP>:8443/cim-ui/scriptConfig.xml and also https://<vCenter Server IP>:8443/cim-ui/index.html

interesting result for https://<vCenter Server IP>:8443/cim-ui/index.html

vcenter 5.1 extensions 2 CIM.PNG

basically - it says it cannot access the hardware monitoring service

Reply
0 Kudos
TBKing
Enthusiast
Enthusiast

I got that same screen when I browsed to the /cim-ui/ sub from my pc...

I can't seem to get to that from a physical windows server - could be firewalls or a number of things particular with my environment.

Do you get the original error when accessing the HW info right from the vCenter server, or your workstation?

Do you use the web client or fat client?

My desktop is on a different subnet from vCenter and not everything worked until I got a number different ports opened between my PC And vCenter AND the Host management network.

I use the fat client.

Reply
0 Kudos
sean0xb8
Contributor
Contributor

I get the same "cannot access" error from all machines including when I run the vSphere client directly on the vCenter server - the action being going to the hardwaer status tab. I also get this error when I type in the specific http address into a browser.

I've get a little more specific error message when I run the index.html file directly on the local vCenter Server - basically the corresponding index.html file under the tomcat directory. A message states that there is a communication error with the server.


Looking at the contents of the file, it seems to run Javascript with an src reference to some js file -- src="com.vmware.vim.cimmonitor.gwt.CimUi.nocache.js". Unfortunately, this is the farthest I could go for now. I'm trying to see if I can somehow determine which source and/or destination port is being used for this communication, but it's hard to tell from a wireshark capture.


I've checked and determined

1. port 8443 is open on the vCenter server.

2. port 5989 is open on ESXi hosts for all other machines

3. port 5988 is open only locally at 127.0.0.1 on the ESXi hosts -- I am guess that the port is open only internally to the local ESXi hosts themselves

4. neither ports 5988 nor 5989 are not open on vCenter server



Additionally,

1. CIM Server service is running on ESXi hosts

2. the Administrators group has necessary permissions to run CIM and etc



Thanks

Reply
0 Kudos
gabrielsimard
Contributor
Contributor


Hi - I was wondering if you or anyone was ever able to figure this one out?

I'm experiencing the exact same issues with all 3 of our ESX hosts since migrating them to esx 5.x, an open ticket with HP for the last month on this hasn't helped either.

My issue is the same, can't get hardware info via WBEM  in SIM, both port 5989 and 5988 indicate open and runing in vsphere, but when i telnet to test the port only 5989 works, 5988 won't connect...  

Reply
0 Kudos
EXPRESS
Enthusiast
Enthusiast

Double check the name of the host you are adding to VC. In my experience I added a few new host and the naming convention was my issue. I had used an underscore "_" in the name once I renamed it with out it all was good.

Thank you, Express
Reply
0 Kudos
gabrielsimard
Contributor
Contributor

No, that is not the issue, I know 100% we've been putting the name in correctly, we've tried this over and over for the last month, both hostname and IP..  for 3 different hosts..  the discovery is succesful, and reports if the system goes down, but does not report any wbem hardware info

Reply
0 Kudos
sean0xb8
Contributor
Contributor

Unfortunately, we were not able to solve this under ESXi 5.1. When we upgraded to 5.5, the issue resolved itself.
The only other suggestion I can think of is to double check the firewall settings under Windows OS itself and see if they match vCenter settings.

Wish I could have been of more help.

Good luck.

Reply
0 Kudos
sgloeckle
Contributor
Contributor

So in general this problem still is not solved :smileycry:

we still have the same problem in various environments with 5.1 and 5.5 even if freshly installed.

anyone some helpful suggestions? nothing had helped so far..

thanks a lot.

Reply
0 Kudos
computerguy7
Enthusiast
Enthusiast

same issue here. has been for a long time.

5.5 vCenter

5.5 and 5.1 hosts.

11 total hosts

no hardware info available on any hosts.

clearly the issue is with vCenter and not with the hosts.

has anyone submitted a VMware ticket on this??

Reply
0 Kudos
sgloeckle
Contributor
Contributor

Hi,

I've created a call at HP for VMware.
Currently all tasks have failed to solve the problems so far.

I'll update here if there is something new...

cheers, Stefan

muetze87
Contributor
Contributor

Same problem here ...

VC 5.0

esxi 5.0/4.1.

Reply
0 Kudos