14 Replies Latest reply on Feb 24, 2010 2:18 PM by brainsphere

    Hyperic unable to detect application's JVM by using JMX plugin

    ravisingh983 Hot Shot
      I was trying to use the JMX plugin in order to monitor one of our application's JVM.
      Followed the steps provided in the link http://support.hyperic.com/display/DOC/JMX+Plugin+Tutorial.
      Though, I was able to connect to the URL service:jmx:rmi:///jndi/rmi://localhost:6969/jmxrmi through JConsole but hyperic is unable to auto-discover the application JVM.
      I did run the commands for testing the plugin but they couldn't match any of the processes running.
      Then I tried running the commands with the argument -Dproc.java.home=<Install path to the applications JDK> and it is able to match the processes running.

      But then how do I deploy the plugin, as the plugin xml does not detect the process until the argument -Dproc.java.home is passed.
      What do I replace this "<property name="PROC_MAIN_CLASS" value="proc.java.home"/>" with.
      I have tried N number of combinations but its still not able to discover the process without passing the -Dproc.java.home argument.
      The tutorial says :
      "replace proc.java.home with the defined name-value pair argument of the process; the MxServer Detector class uses this variable to perform auto-discovery"

      Thanks,
      Ravi
        • 1. Re: Hyperic unable to detect application's JVM by using JMX plugin
          ravisingh983 Hot Shot
          Does any one actually know anything in Hypeic User forums about JMX plugins. As I see another discussion with subject "Why my simple JMX plugin doesn't work? " has not been answered which was posted long back.
          Any help will be highly appreciated.
          • 2. Re: Hyperic unable to detect application's JVM by using JMX plugin
            ravisingh983 Hot Shot
            Has anyone faced similar problem before. If yes, please do leave a solution for the same.
            • 3. Re: Hyperic unable to detect application's JVM by using JMX plugin
              navechou Hot Shot
              Hi Ravi,

              Just to understand your problem:
              Which one of the following have you used: (You should use one)
              * <property name="PROC_HOME_ENV" value="JAVA_HOME"/>
              * <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>
              * <property name="PROC_MAIN_CLASS" value="proc.java.home"/>

              How you have defined:
              option name="process.query" in config file.

              Regards,
              Naveen
              • 4. Re: Hyperic unable to detect application's JVM by using JMX plugin
                ravisingh983 Hot Shot
                By config file, i think you mean the plugin xml.
                I have tried with all three properties (
                * <property name="PROC_HOME_ENV" value="JAVA_HOME"/>
                * <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>
                * <property name="PROC_MAIN_CLASS" value="proc.java.home"/>)
                individually but none of them worked.

                I have defined option name="process.query" in config file.

                One thing I am not able to understand is what do i replace proc.java.home in the above properties.

                In addition to that I tried running it with only PROC_MAIN _CLASS property which looked something like this :
                <property name="PROC_MAIN_CLASS" value=com.test.AppLoader/> but it gave an error that PROC_HOME_PROPERTY is not defined.

                Can you please provide me with a sample of the plugin in case you were able to run it with your application?
                Thanks for the help.
                • 5. Re: Hyperic unable to detect application's JVM by using JMX plugin
                  navechou Hot Shot
                  Hi Ravi,

                  e.g.
                  In my configuration to monitor Resin:
                  <property name="PROC_MAIN_CLASS" value="com.caucho.server.resin.Resin"/>
                  <property name="PROC_HOME_PROPERTY" value="resin.home"/>

                  In your configuration to monitor AppLoader it would be:
                  <property name="PROC_MAIN_CLASS" value=com.test.AppLoader/>
                  <property name="PROC_HOME_PROPERTY" value="app.home"/>

                  Change app.home with your application home.

                  I hope it will help.

                  Cheers!!!
                  Naveen
                  • 6. Re: Hyperic unable to detect application's JVM by using JMX plugin
                    ravisingh983 Hot Shot
                    1. In your example as you suggested I changed

                    <property name="PROC_MAIN_CLASS" value="com.test.TestLoader"/>
                    <property name="PROC_HOME_PROPERTY" value="/ais_local/share/ravi/testInstall/jdk"/>
                    here "/ais_local/share/ravi/testInstall/jdk" is the install path of JDK for the application.

                    but it doesn't detect the process on running
                    java -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug

                    Here JMX-metrics is the name of the plugin

                    2. Then i commented the PROC_HOME_PROPERTY property tag and ran the same command

                    DEBUG [main] [PluginDiscoverer] Adding Auto Scanners=[ServerSignature {stName=Java 1.5.x, match=[null], exclude=[null], regMatch=[null]}, ServerSignature {stName=Java 1.6.x, match=[null], exclude=[null], regMatch=[null]}]
                    DEBUG [Thread-1] [AutoScan] Running AutoServerDetector for: Java 1.5.x
                    DEBUG [Thread-1] [MxServerDetector] State.Name.sw=java,Args.*.eq=com.test.RaviLoader matched 6 processes
                    ERROR [Thread-1] [AutoScan] AutoScan failed for Java 1.5.x
                    java.lang.NullPointerException
                            at org.hyperic.hq.product.jmx.MxServerDetector.isMatch(MxServerDetector.java:160)
                            at org.hyperic.hq.product.jmx.MxServerDetector.getServerProcessList(MxServerDetector.java:251)
                            at org.hyperic.hq.product.jmx.MxServerDetector.getServerResources(MxServerDetector.java:309)
                            at org.hyperic.hq.autoinventory.scanimpl.NullScan.scan(NullScan.java:83)
                            at org.hyperic.hq.autoinventory.Scanner.start(Scanner.java:196)
                            at org.hyperic.hq.autoinventory.ScanManager.mainRunLoop(ScanManager.java:141)
                            at org.hyperic.hq.autoinventory.ScanManager.access$000(ScanManager.java:41)
                            at org.hyperic.hq.autoinventory.ScanManager$1.run(ScanManager.java:107)
                    DEBUG [Thread-1] [AutoScan] Running AutoServerDetector for: Java 1.6.x
                    DEBUG [Thread-1] [MxServerDetector] State.Name.sw=java,Args.*.eq=com.test.RaviLoader matched 6 processes
                    ERROR [Thread-1] [AutoScan] AutoScan failed for Java 1.6.x
                    java.lang.NullPointerException
                            at org.hyperic.hq.product.jmx.MxServerDetector.isMatch(MxServerDetector.java:160)
                            at org.hyperic.hq.product.jmx.MxServerDetector.getServerProcessList(MxServerDetector.java:251)
                            at org.hyperic.hq.product.jmx.MxServerDetector.getServerResources(MxServerDetector.java:309)
                            at org.hyperic.hq.autoinventory.scanimpl.NullScan.scan(NullScan.java:83)
                            at org.hyperic.hq.autoinventory.Scanner.start(Scanner.java:196)
                            at org.hyperic.hq.autoinventory.ScanManager.mainRunLoop(ScanManager.java:141)
                            at org.hyperic.hq.autoinventory.ScanManager.access$000(ScanManager.java:41)
                            at org.hyperic.hq.autoinventory.ScanManager$1.run(ScanManager.java:107)
                    0 servers detected

                    3. Then i commented the PROC_MAIN_CLASS property tag and uncommented PROC_HOME_PROPERTY and wrote it as given in the JMX plugin example

                    <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>
                    Then I ran
                    java -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug

                    This command deducted 0 processes

                    4. Then I ran the command
                    java -Dproc.java.home=/ais_local/share/ravi/testInstall/jdk -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug

                    On running this it gave me an output 1 processes matched with the metrics.

                    Now obviously i can't do this as there is no way i should be passing the same in hq-agent.sh other than modifying <Hq-agent-Install>/bundles/agent-4.1.2-1053/bin/hq-agent.sh which I should not.
                    • 7. Re: Hyperic unable to detect application's JVM by using JMX plugin
                      navechou Hot Shot
                      Hi Ravi,

                      What I understood from your trace:
                      1. <property name="PROC_HOME_PROPERTY" value="/ais_local/share/ravi/testInstall/jdk"/> should be <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>

                      2. Not able to find the process for "com.test.TestLoader".
                      3. There is no java process to look with system JAVA_HOME or no JAVA_HOME found.
                      4. There is one process found at given JAVA_HOME.

                      Before you carry-on as you suggested, Try following:

                      * Look at the JAVA_HOME setting.
                      * Set your JAVA_HOME with the script, you are invoking your process. e.g
                      JAVA_HOME=/ais_local/share/ravi/testInstall/jdk
                      export JAVA_HOME
                      * Apply following settings:
                      <property name="PROC_MAIN_CLASS" value="com.test.TestLoader"/>
                      <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>

                      Further I am interesting in how you have defined:
                      option name="process.query" in config file.

                      Cheers!!!
                      Naveen
                      • 8. Re: Hyperic unable to detect application's JVM by using JMX plugin
                        ravisingh983 Hot Shot
                        I tried what you had stated at the end of your mail i.e.

                        * Look at the JAVA_HOME setting.
                        * Set your JAVA_HOME with the script, you are invoking your process. e.g
                        JAVA_HOME=/ais_local/share/ravi/testInstall/jdk
                        export JAVA_HOME
                        * Apply following settings:
                        <property name="PROC_MAIN_CLASS" value="com.test.TestLoader"/>
                        <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>

                        Did all of the above but the commands below do not match a single process now.

                        * java -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug
                        or
                        * java -Dproc.java.home=/ais_local/share/ravi/testInstall/jdk -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug


                        In addition this is how the option name process.query is being passed

                        <option name="process.query" description="PTQL for Java Process" default="State.Name.eq=java,Args.*.ct=proc.java.home"/>
                        • 9. Re: Hyperic unable to detect application's JVM by using JMX plugin
                          navechou Hot Shot
                          Hi,

                          There seems to be problem with PTQL.
                          Try changing it to
                          • 10. Re: Hyperic unable to detect application's JVM by using JMX plugin
                            ravisingh983 Hot Shot
                            Had tried all these options earlier to no effect :-). Did try the same now but to no effect again.
                            • 11. Re: Hyperic unable to detect application's JVM by using JMX plugin
                              ravisingh983 Hot Shot
                              Now say I comment out the property PROC_MAIN_CLASS and run the command

                              java -Dproc.java.home=/ais_local/share/ravi/testInstall/jdk -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug

                              Then i get the output 1 process matched and output as in the file attached

                              but if i run the command like

                              java -jar /opt/hyperic/hyperic-hq-agent/bundles/agent-4.1.2-1053/pdk/lib/hq-product.jar -Dplugins.include=JMX-metrics -m discover -a metric -Dlog=debug

                              I get 0 processes matched.
                              • 12. Re: Hyperic unable to detect application's JVM by using JMX plugin
                                navechou Hot Shot
                                It's really something not configured in order. Can you share your plugin.xml and how you are running your java program.

                                ~Naveen
                                • 13. Re: Hyperic unable to detect application's JVM by using JMX plugin
                                  ravisingh983 Hot Shot
                                  This is how we launch the testLoader

                                  java -Xms512m -Xmx1024m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=6969 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dvendor=noapp -DvendorFile=/ais_local/share/ravi/testInstall/properties/servers.properties -classpath /ais_local/share/ravi/testInstall/jar/bootstrapper.jar -verbose com.test.TestLoader -f /ais_local/share/ravi/testInstall/properties/dynamicclasspath.cfg.

                                  This is called through a startup script of the application and does a lot of initialisation of application entities properties and stuff.
                                  The plugin is attached
                                  • 14. Re: Hyperic unable to detect application's JVM by using JMX plugin
                                    brainsphere Hot Shot
                                    Having the same problem, I found a configuration that at least works:

                                    In the *-plugin.xml provide the following settings, substituting *only* "your.main.class.Main" (I think your main class was "com.test.TestLoader"), do *not* change "proc.java.home":

                                    :
                                    <!-- derive installpath from JAVA_HOME env prop... -->
                                    <property name="PROC_HOME_ENV" value="JAVA_HOME"/>

                                    <!-- derive installpath from -Dproc.java.home=... -->
                                    <property name="PROC_MAIN_CLASS" value="your.main.class.Main"/>
                                    <property name="PROC_HOME_PROPERTY" value="proc.java.home"/>
                                    :


                                    Then start *your* program with

                                    java ...... -Dproc.java.home=<YOUR_JAVA_HOME> ..., substituting YOUR_JAVA_HOME with the path to your jdk, which would give you the command line:

                                    java -Xms512m -Xmx1024m -Dproc.java.home=<YOUR_JAVA_HOME> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=6969 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dvendor=noapp -DvendorFile=/ais_local/share/ravi/testInstall/properties/servers.properties -classpath /ais_local/share/ravi/testInstall/jar/bootstrapper.jar -verbose com.test.TestLoader -f /ais_local/share/ravi/testInstall/properties/dynamicclasspath.cfg

                                    This way process discovery works (at least with my installation ;-)).

                                    I agree, this is a bit awkward or at least the documentation of HQ needs some rework to clarify this.