9 Replies Latest reply: Mar 19, 2007 9:58 PM by clee201110141 RSS

    How can I programmatically disable an alert

    mbaker3 Hot Shot
      We are building a self-service portal for our customers to allow them to do simple management of their hosted environment. For instance they want to be able to restart a WebLogic managed server. However when they do this it will cause Hyperic to raise an alert as the server will become unavailable.

      Is there some way that my restart routine can disable the Hyperic alerts for this managed server while the restart takes place? I don't see an API for this but I assume we could modify the database to set the alerts to disabled and then back to enabled.

      Thanks . . . Murray
        • 1. Re: How can I programmatically disable an alert
          Expert
          Hi Murray,

          The concept of maintenance windows is something that we are thinking 
          about developing for future versions of HQ.  Currently, there's no 
          way to "temporarily disable" an alert programmatically.  However, you 
          can achieve this by using the some additional configurations to the 
          alert.  If you set the alert to fire for an unavailability of at 
          least the duration of the restart, rather than every time, then you 
          won't have false alerts firing while you are restarting the server.

          You should not modify the database to affect this behavior.  Object 
          data from the database is usually cached and maintained in our 
          application server, and you have to guarantee that a direct 
          modification to the database will be read by the HQ application.

          Charles



          • 2. Re: How can I programmatically disable an alert
            mbaker3 Hot Shot
            Charles,

            Thanks. The maintenance window idea would be very useful and would solve part of the problem. That is something we would definitely use.

            We are also looking at ad-hoc situations which are a bit trickier. The problem with extending the duration before you alert is, of course, that you delay the alert when a real problem happens. It would be useful to extend the concept of a maintenance window a little further. For example to set the alert differently for different times of day (business hours vs overnight). That way we could have a short time during the day when we can investigate it easily and longer at night when it is less critical and we have to get people out of bed.

            Thanks for your help. In the short term we may just have to go with the extended duration.

            Cheers . . . Murray
            • 3. Re: How can I programmatically disable an alert
              mbaker3 Hot Shot
              Charles,

              If we were to use the Hyperic control function to perform the self-service start/stop would that help? That is, if I stop WebLogic via the control function does it, or can it be made to, disable the alerts?

              Thanks . . . Murray
              • 4. Re: How can I programmatically disable an alert
                Expert
                Hi Murray,

                That isn't a built in function of the product.  This certainly sounds like an interesting feature.  In all honesty, it's probably not a difficult action to add to the product.  We have a concept of "Recovery Alert" in the Enterprise product that does the opposite of what you are detailing.  You'd actually set up another alert that could watch the control functions being performed and "enable" another alert when that happens.  However, I just don't see a way of setting this up so that you can disable an alert temporarily through an automated process.

                Charles
                • 5. Re: How can I programmatically disable an alert
                  mbaker3 Hot Shot
                  Charles,

                  For my purposes the shell would be a perfectly good way to do it. Currently you have "alertdef list",  "alertdef add" and  "alertdef delete". What I need is  "alertdef modify" with the ability to enable/disable the alert. So then when I'm stopping the resource I disable the alert(s) and when I restart it I enable the alert.

                  On the same subject I have looked at the doco at http://support.hyperic.com/confluence/display/DOCSHQ30/Alertdef+CLI+Command. Is it possible to do the add via a single command or is it only available via the dialogue? If there is a single line command then I could delete the alert and add it again. (Crude but I guess it would work.)

                  Thanks . . . Murray
                  • 6. Re: How can I programmatically disable an alert
                    Expert
                    Hi Murray,

                    So you'd rather do this manually rather than getting an alert that 
                    tells you if something was restarted?

                    I suppose adding a shell command to enable/disable alert definition 
                    wouldn't be too difficult.   I wouldn't imagine that you'd want to 
                    delete the alert and add it again because of the number of properties 
                    you will have to set.

                    Charles



                    • 7. Re: How can I programmatically disable an alert
                      mbaker3 Hot Shot
                      Charles,

                      In our case it wouldn't really be manual. Turning the alert on again would be done in our management framework once it determined that WebLogic was back up. That way we can let our customers start/stop their own environments without an alert coming to us.

                      What we need is just a function to disable the alert and then enable it again, as you can do through HQAdmin.

                      In the long term product development it should be made possible to do anything through the shell that can be done through HQAdmin. That would allow is to fully automate the monitoring set-up to complement our automation of things like WebLogic domain builds.

                      Cheers . . . Murray
                      • 8. Re: How can I programmatically disable an alert
                        On Fri, 2007-03-16 at 17:18 -0700, Charles Lee wrote:
                        > Hi Murray,
                        >
                        > So you'd rather do this manually rather than getting an alert that 
                        > tells you if something was restarted?
                        >
                        > I suppose adding a shell command to enable/disable alert definition 
                        > wouldn't be too difficult.   I wouldn't imagine that you'd want to 
                        > delete the alert and add it again because of the number of properties 
                        > you will have to set.
                        >
                        > Charles
                        >

                        I concur that to be able to enable/disable the alerts through the shell
                        could be very useful. I assume (but haven't looked into it yet) that I
                        can check the status (enabled/disabled) of alerts through the shell, as
                        well as list (in some way) all the alert definitions (if not, that could
                        be VERY useful as well, to check for consistency or misconfigurations).

                        Also agree that EVERYTHING that can be done through the gui should be
                        available in the shell. Sometimes when things happen we just don't have
                        direct access to a web browser and/or fast connection.

                        Just suggestions, not complaints!

                        John



                        • 9. Re: How can I programmatically disable an alert
                          Expert
                          Okay, I will look into adding that functionality into the CLI.  Although, EVERYTHING that the GUI can do in the CLI may be next to impossible, though :-(.

                          Charles