Few eval questions-monitoring over firewall,remote checks,plugins

[jooray]

Hello,

I am currently evaluating Hyperic. We currently use Mon and Zabbix for network monitoring of cca 200 machines (Solaris, Linux, FreeBSD and few Windows boxes). We also evaluated Pandora and Nagios, but we got rid of them. It is not that easy to migrate all those monitors to different system, but I am pretty convinced, that Hyperic HQ would be great upgrade. I have a few questions though:

1.) We monitor several networks of cca 30-40 machines, which are using private address space. We would like to have one centralized monitoring. In documentation I saw, that server also connects back to agents. What is this for? Is it possible to have only agent->server connection? It is not a problem to let agents behind NAT connect outside to the central monitoring server, but connecting the other way is more difficult.

Of course we could build another server, but it is nice to have centralized monitoring. Is it possible for example to have parent-child server relationship (so child server, which will be inside that network will report and communicate with central parent server?).

2.) Is it possible to define remote checks for server? For example in mon, we currently have check, that will send e-mail to server, connect to it's pop3 port, login, download the message and compare body hash and then delete the message. It is done remotely (currently from central mon machine), but it would be nice to assign this monitor to a particular machine. (BTW: does Hyperic have some check? We found it quite useful, since it does not only monitor a running service, but it shows, that "e-mail works for users". Non working could be cause by dozen different problems, which are more difficult to check, i.e. antivirus software working, correct DNS MX records, ...).

3.) Is it possible to write plugins in Java (or any other language)? I looked for examples of plugins, but they are mostly some kind of XML description. I would for example like to add CIFS monitoring support of Samba, NFS server monitoring using recently open-sourced Sun's NFS client library, etc.

Thanks,

Juraj.

[admin]

Hi Juraj,

The agent's require bi-directional communication because as resources are configured from the HQ portal we need to push that configuration out to the agent responsible for monitoring that resource. 99% of the traffic is from Agent to Server though.

The issues around running multiple HQ servers in different data centers has been coming up more frequently lately. We evaluating the possibility of having a master server that would roll up the data from the various servers. If you have other suggestions let us know so we can incorporate that feedback.

For #2, we do allow a variety of remote checks. The checks can be performed from any host in your environment. A full list can be found in our documentation:

http://support.hyperic.com/confluence/display/DOCSHQ30/Network+Services

For #3, we support plugins written in Java. In fact, the majority of our plugins are Java based. The XML descriptor language is merely for simplicity in creating new plugins that may use a common metric collection method. (e.g. SNMP, Windows PerfMon, JMX, scripts, etc). The documentation on the Java portion is a bit light though, so feel free to post any questions you have to the forums.

For the products you mention, I assume there is some sort of command line utility that can be used to gather the metrics? If that's the case the script plugin would be the quickest way to implement support for those products.

Hope that helps,
-Ryan

[sandrea08]

Hi Juraj,

Thanks for this Information. I am evaluating Hyperic for monitoring Installations of a custom Application at a number of customer sites. In this scenario we absolutely need a central Server - but we cannot have under any circumstances get ingoing holes punched into the firewalls of each customer's datacenter. On the other hand, uploading by https from the client to the server is fine.

Can Hyperic be used at all under these circumstances. Which functions would be affected, if it works at all?

A master server would not exactly serve our purpose, as I do not want to install a Hyperic Server at sites, where I monitor perhaps only one machine. Would it be possible in the future to leave all communication to be initiated by the agent? i.e., it could regularly check with the server, whether there is new information for the agent.

Best Regards,
Stefan

Message was edited by: s.andreatta@synedra.com

[jduino]

Unfortunately you really need both directions. You need to get the info
from client to server, but the server is what controls/configures the
client.

One of my favorite quick/easy solutions for things such as this is to
build tunnels using SSH. A VPN of some other sort could be used, too.

John

On Thu, 2007-08-02 at 01:31 -0700, Stefan Andreatta wrote:
> Hi Juraj,
>
> Thanks for this Information. I am evaluating Hyperic for monitoring Installations of a custom Application at a number of customer sites. In this scenario we absolutely need a central Server - but we cannot have under any circumstances get ingoing holes punched into the firewalls of each customer's datacenter. On the other hand, uploading by https from the client to the server is fine.
>
> Can Hyperic be used at all under these circumstances. Which functions would be affected, if it works at all?
>
> A master server would not exactly serve our purpose, as I do not want to install a Hyperic Server at sites, where I monitor perhaps only one machine. Would it be possible in the future to leave all communication to be initiated by the agent? i.e., it could regularly check with the server, whether there is new information for the agent.
>
> Best Regards,
> Stefan

[kgorman]

I just wanted to chime in, I have the same request. I would like to have an agent that can optionally run w/o any communication back from the server. Our colocation/firewall setup is such that servers don't allow communications into the datacenter, but outbound is ok for a few select services.

For instance:

client -> server

and not:

client <-> server

[VoodooZ]

Shouldn't be an issue if the communication is TCP based right?
I guess it depends who initiates the TCP connections first?
Unless you have very restrictive firewall rules...

[kgorman]

yes, firewall rules is the issue.

[bclark_hyperic]

I would like to make this a feature request. It would be great (and better from a security perspective) if the agents could poll the server every X seconds/minutes for changes in their configuration rather than have the server push the config's out to them. This would increase the amount of network traffic between the client and server, but I would assume that you could do this "new config check" with just a couple of packets of network data, having almost no impact on performance.

[lrpv]

That feature request is interesting. We're in the same situation and are evaluating both this and another security tool with the same architectural issue -- that of punching firewall holes from the server to multiple clients split across separate networks/datacentres/customers. We can technically do it, but the firewall rules get very messy and are susceptible to trunk network connections going down.

I would also vote for the tiered feeder setup, where one master server acts a console, and other 'regional master servers' pass data into it.

[lucrasoft]

i'm in exactly the same situation : evaluating Hyperic, very excited about the product, but .... not with the architect.

with the current agents<->server connection requirement (two way), it's kind of "useless" in our envirorment; we want to monitor a lot of private remote networks ; there networks are remote , not reachable from server point of view. it's impossible to open all routes / connections .

some suggestions :

1. agent keeps their tcp connection to server open. server uses the always-on-tcp-connection of the connected "agent" when it needs to send commands / data.

2. server has a "queueu" of commands / request. Agents connects regular and checks the "queue" ..

3. some sort of 3rd component in the hyperic architecture : the HQ proxy server. A component between agent - proxy - server.

Jooray : we use zabbix too, what was your reason to change ? did you change already ? did you find another solution ?

[jtravis_hyperic]

It seems that this type of scenario is common, and something that
we'll be discussing for inclusion in the next release.

The architecture has held up very well for the past few years, but
since we are seeing more and more sizeable deployments, with more
complex requirements, our architecture will need to change and
accommodate.

We'll likely be getting some combination of agent->server only
communication (with always-on-tcp), as well as proxy support, which
should give everyone what they need. In the meantime, allowing the
server to connect to the agent is a pretty simple firewall rule --
I'm not sure why people fight it so much.

-- Jon



On Oct 27, 2007, at 2:49 PM, Lucas Vos wrote:

> i'm in exactly the same situation : evaluating Hyperic, very
> excited about the product, but .... not with the architect.
>
> with the current agents<->server connection requirement (two way),
> it's kind of "useless" in our envirorment; we want to monitor a
> lot of private remote networks ; there networks are remote , not
> reachable from server point of view. it's impossible to open all
> routes / connections .
>
> some suggestions :
>
> 1. agent keeps their tcp connection to server open. server uses the
> always-on-tcp-connection of the connected "agent" when it needs to
> send commands / data.
>
> 2. server has a "queueu" of commands / request. Agents connects
> regular and checks the "queue" ..
>
> 3. some sort of 3rd component in the hyperic architecture : the HQ
> proxy server. A component between agent - proxy - server.
>
> Jooray : we use zabbix too, what was your reason to change ? did
> you change already ? did you find another solution ?

[sandrea08]

> In the meantime, allowing the
> server to connect to the agent is a pretty simple
> firewall rule -- I'm not sure why people fight it so much.
>
> -- Jon

True, the rule is simple - getting it implemented on a firewall that is not yours may be simple or it may be impossible, depending on security policy. With most of my customers there is no way I will ever get such an access to their network.

[lucrasoft]

uhm.

opening one port, isn't really an issue in my case (although : some customers would not allow changes in their router/firewall configuration).

my situation : all my customers are always behind a NAT router. they all have a private address range inside the lan and 1 public ip address outside..i think this situation is quite common.. imho to make it possible for the hyperic server to directly connect to a hyperic agent, i have to setup a portforwaring rule for every client !!! thát's the reason i fight it so much 😉

you were talking about next releases etc.. is there a roadmap / planning ??
thanx.

[JonB_hyperic]

As well as having to port forward individual ports to agents behind a nat firewall you also have to modify each agent to set agent.ListenPort=xxxx

It starts becoming a real pain.

I too have come from a Zabbix background and while I got frustrated with the way Zabbix development was happening (or not happening) I miss the ability to set an agent mode as Active ( agent -> server) or Passive ( server -> agent)

One disadvantage of a Hyperic agent in agent -> server mode would be that Controls will not happen in real time or perhaps not work at all.

Another option would be Distributed Node Monitoring i.e node servers report all resources/metrics back to a Master or Central server. Node servers can be configured via the Master or Central server.

I am currently evaluating Hyperic as a Managed Service Platform for my business. I am monitoring several clients network, Linux servers, Windows servers and desktops. I have set Controls that will run scripts to force the remote machine to create a remote uVNC single click connection via a repeater to a VNC viewer and run my hardware/software/patch status inventory script regularly on each machine (results reported back remotely to a seperate database and web frontend). All alerts are sent via email to my trouble ticket system.

The whole lot is open source.

Jon

[lucrasoft]

question to the Hyperic people :

about this remote locations / nat / firewall request

- what's the roadmap ?
- can we make an official request somehow ?
- could someone give us a reaction about the plans ?

thank you !

[jtravis_hyperic]

Hey Lucas,

Well, since this is an Open Source project, you're free to make the
request right here on the users list. An even better option would be
to file a JIRA enhancement request in our issue tracker
(jira.hyperic.com)

It seems that there are a good amount of people requesting more
flexibility in this area. We're coming up with the final roadmap of
features for 4.x (our mid-next-year-release), so we'll be sure to let
you know what we're working on, and will likely be chatting about it
with the users to make sure we meet everyone's needs.

-- Jon



On Nov 1, 2007, at 4:20 PM, Lucas Vos wrote:

> question to the Hyperic people :
>
> about this remote locations / nat / firewall request
>
> - what's the roadmap ?
> - can we make an official request somehow ?
> - could someone give us a reaction about the plans ?
>
> thank you !

[solodan]

If I may just chime-in,
I am the process of evaluating Hyperic. We are also
currently using Zabbix (passive checks).

I am also in a situation where server --> agent
communications is a show stopper.

I am otherwise thrilled with the featureset.
Thanks for your openess to these architectural options

[staceyeschneide]

All good questions:

1. We are getting close to announcing the roadmap for 3.2 which will be coming out later this year. Stay tuned...
2. For Enterprise customers, enhancement requests can be formally entered using your JIRA login, for community, well - you just did! There is also a product roadmap forum which sort of collects these feature requests, but the forums are the best way to make your requirements/ideas known.
3. I am not sure what reaction you would like, Travis (one of our core developers) mentioned above that we were working on our roadmap for the 4.0 release (scheduled for summer) which this is being considered. We'll communicate something on 4.0 as soon as we have firm plans.

[mattm_hyperic]

Was the Server to Client communication solved in the latest release? Looking to monitor nodes behind strict firewalls as well...

[excowboy]

Hi Matt,

no there where no significant changes in the communication mode.

Mirko