Hi,
I'm running hyperic on HP x86 blades with a 64-bit Solaris OS.
The hyperic-agent service keeps entering maintenance state with the following logs;
xhkny91035>svcs -a | grep hyper
online Jan_23 svc:/ericsson/eric_3pp/hyperic-server:default
maintenance 11:45:55 svc:/ericsson/eric_3pp/hyperic-agent:default
Look at service log
xhkny91035>svcs -x /ericsson/eric_3pp/hyperic-agent
svc:/ericsson/eric_3pp/hyperic-agent:default (Hyperic Agent)
State: maintenance since Fri Mar 15 11:45:55 2013
Reason: Start method failed repeatedly, last died on Killed (9).
See: Hyperic Agent-4.6(1)
See: /var/svc/log/ericsson-eric_3pp-hyperic-agent:default.log
Impact: This service is not running.
xhkny91035>tail /var/svc/log/ericsson-eric_3pp-hyperic-agent:default.log
[ Mar 15 11:40:54 Leaving maintenance because clear requested. ]
[ Mar 15 11:40:54 Enabled. ]
[ Mar 15 11:40:54 Executing start method ("/opt/hyperic-agent/ericsson/bin/hq-service.sh start") ]
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Oracle Corporation SunOS 5.10 Generic Patch January 2005
-n Starting HQ Agent...
[ Mar 15 11:45:55 Method or service exit timed out. Killing contract 25776 ]
[ Mar 15 11:45:55 Method "start" failed due to signal KILL ]
xhkny91035>tail -20 /var/svc/log/ericsson-eric_3pp-hyperic-agent:default.log
[ Jan 23 15:57:57 Disabled. ]
[ Jan 23 16:05:51 Enabled. ]
[ Jan 23 16:05:51 Executing start method ("/opt/hyperic-agent/ericsson/bin/hq-service.sh start") ]
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Oracle Corporation SunOS 5.10 Generic Patch January 2005
-n Starting HQ Agent...
[ Jan 23 16:10:51 Method or service exit timed out. Killing contract 5607 ]
[ Jan 23 16:10:51 Method "start" failed due to signal KILL ]
[ Mar 15 11:40:54 Leaving maintenance because clear requested. ]
[ Mar 15 11:40:54 Enabled. ]
[ Mar 15 11:40:54 Executing start method ("/opt/hyperic-agent/ericsson/bin/hq-service.sh start") ]
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Oracle Corporation SunOS 5.10 Generic Patch January 2005
-n Starting HQ Agent...
[ Mar 15 11:45:55 Method or service exit timed out. Killing contract 25776 ]
[ Mar 15 11:45:55 Method "start" failed due to signal KILL ]
Look at agent log
xhkny91035>tail /var/log/hyperic/agent/agent.log
...
java.lang.OutOfMemoryError: Java heap space
at java.util.TreeMap.put(TreeMap.java:554)
at java.util.TreeSet.add(TreeSet.java:238)
at org.hyperic.util.file.DiskList.genFreeList(DiskList.java:250)
at org.hyperic.util.file.DiskList.<init>(DiskList.java:141)
at org.hyperic.hq.agent.server.AgentDListProvider.intrCreateList(AgentDListProvider.java:100)
at org.hyperic.hq.agent.server.AgentDListProvider.createList(AgentDListProvider.java:111)
at org.hyperic.hq.measurement.agent.server.SenderThread.<init>(SenderThread.java:131)
at org.hyperic.hq.measurement.agent.server.MeasurementCommandsServer.startup(MeasurementCommandsServer.java:232)
at org.hyperic.hq.agent.server.AgentDaemon.startHandlers(AgentDaemon.java:796)
at org.hyperic.hq.agent.server.AgentDaemon.start(AgentDaemon.java:917)
at org.hyperic.hq.agent.server.AgentDaemon$RunnableAgent.run(AgentDaemon.java:1014)
at java.lang.Thread.run(Thread.java:662)
...
Here's the memory dimensioning
I've had a look at the following after searching on the web.
1-Insufficient memory allocation in hyperic-server configuration file
2-Problems with JVM garbage collection
3-Problems with hyperic plugins being greedy with memory
Need to perform further analysis
-restart hyperic-server and activate garbage collection logging to see rate of memory usage over few days
According to VMware Vfabric Hyperic Server 4.6 documentation:
If you are running Hyperic Server on a 64-bit system with 4GB (4096 MB) or less memory, Hyperic recommends you use 32-bit JVM. A 64-bit JVM is not recommended unless you have more memory. You might need twice as much heap on a 64-bit system as on a 32-bit system to achieve the same performance.
The Model MWS is a 64-bit system with 16GB RAM and 64-bit JVM. Satisfactory.
