An honest opinion required...

[zaspam]

Hi all,

I'm considering replacing my current syslog server (Kiwi syslog) with Log Insight.

I would like to hear an honest opinion on how adequate Log Insight is for a syslog replacement.

I am currently collecting events from a multitude of network devices (Cisco, Fortinet, etc.) a number of Windows Servers, Out-Of-Band management solutions (HP ILO, Dell IDRAC) and a small number of Linux VMs.

I export and archive events after 6 months onto an SMB share.

I would like to hear if Log Insight is a suitable solution, how is it licensed, what are the possible caveats, any advantages or disadvantages.

Grateful in advance,

[jpsider]

We did an investigation a while back, and the reason we did not go with Loginsight at the time, was due to the limitation on creating custom queries. I'm not sure of the current status, but If you want to create custom queries, be sure to research that.

[sflanders]

jpsider -- can you elaborate on what you mean by "creating custom queries"?

[jpsider]

I will caveat with, "when we reviewed the product" I am unaware of any recent releases/changes to the tool.

now on to the good stuff. When we had log insight ingest a bunch of logs, we were unable to correlate data across different types of infrastructure.  At the time log insight was geared for retrieving logs/messages from vmware, not something from my application or infrastructure.  Basically there was no query language available, for me to search on specific things, across different types of logs.

[zaspam]

So in your experience you would say that LogInsight is more VMware specific rather than a generalist solution?

Have you tried collecting logs from network devices?

[sflanders]

jpsider -- so by custom queries you mean query language. What query where you trying to build? Note LI has a natural language query language and works similar to a database (though it is not) -- you have search with filters (think database constraints), functions and groupings (think group by).

Can you also elaborate what you mean by correlate? Correlate by what?

[sflanders]

Note that Log Insight supports ingestion from syslog -- which most systems support today (VMware, non-VMware, physical, virtual, etc). As of LI 2.0 it also supports ingestion API (which can be used by the LI agent or anyone). If you are interested in additional ways to ingestion events into LI, please let us know!

[sflanders]

zaspam -- LI is definitely not VMware specific, though it is best for VMware. What I mean is that it has native integration for vSphere and vR Ops, but it supports syslog and you will find a variety of content packs for physical (as well as virtual) network devices in the marketplace: VSX Home | Solution Exchange including Cisco, Juniper, Brocade, Dell, Arista, Extreme, etc. I hope this helps!

[jpsider]

I will say that in our review of the product for our production environment it was not going to meet our needs at that time.  I cannot argue that to be the case at this time, we have not reviewed recent releases of the product.

When we reviewed the product we were interested in vmware logs and our custom java application logs.

[jpsider]

Sure, We have a custom Java application that interacts with VMware, during our evaluation of the product nearly 2 years ago, we were unable to correlate events from VMware, with evens from our Java application.  Part of that problem was log normalization, Our timestamp was different than the format of VMware, but in theory that should be over come able, by standarizing our logs.  But again, at the time it was not user friendly. I assume by your statements that the product has improved.

[sflanders · Accepted solution]

Thanks jpsider -- your insights are very helpful. I am just trying to ensure I understand the feedback because words/phrases like "custom queries" and "correlation" mean different things to different people Technically, both are possible in LI, but of course it depends on the specific use-cases and what features you are aware of / tried. For example, LI already normalizes timestamps (and has since version 1.0) so different formats between VMware logs and Java applications do not matter. For example see this post: Time in Log Insight: Events + Timestamps + Queries - SFlanders.net. In addition, LI has improved over time and now natively supports a timestamp parser on the client via the LI agent: Log Insight 3.0 Agents: Timestamp Parser - SFlanders.net. Long story short, I agree the product has evolved since you last tried it, but I also believe at also had capabilities such as custom queries and correlation. Of course, none of this is to say it fit your particular use-cases!

[MichaelRyom]

Hi

Just want to give my opinion... Around three years back I had a project in collecting log data from datacenter components. Working as part of a core datacenter team, my responsibility were to the VMware platform, but the project was not centered around VMware, but the entire datacenter core infrastructure components. There were two reasons collect data, first was security, we needed a way to search through, events when ever a security incident was raised. Secondly, log would be used for operational purposes, tracking, finding events, making correlations between different datacenter silos.

The networking team were already using Splunk, but due to an storage issue which generated millions of events/alerts in vCenter, where we tried to send logs to the network teams Splunk installation and got it to create a huge backlog of data not being indexed. We had learned two things, first the amount of data that a VMware platform could generate can be quite large and with major incidents the amount of data being logged could grow very much and the second thing was that buying Splunk license would cost us more than a million looking a list prices. Given that we might have to buy more capacity up front than what was needed on a daily basis to account for growth during major incidents, as this would be where a sysloging solution would add most value, it seem very expensive and hard to predict how large a license we would need to buy, to always have data at hand.

But more so this would be a general issue and not so much a VMware issue. Think DDOS, worms, malware etc. All thing the could grow the number of events though the roof and over our license limit to index data.

Given that most commercial syslog solutions are priced, licensed and build on the same open source tools, all where skipped (by the way Kiwi syslog, solarwinds, vCenter syslog etc, are in my book useless in this context, due to the horrible search capabilities). Instead a solution called ELSA - GitHub - mcholste/elsa: Enterprise Log Search and Archive was chosen. The reasons were, high ingestion rate/scalable, simple GUI which was easy to use and lastly pricing - free. The thought was that the money saved on a commercial license could easily make up for some of the work that needed to be done to make I useful in our context.

During the POC of ELSA, VMware release the first technical preview of Log Insight, naturally being a VMware guy I jumped in and tested it. Shortly after I started advocating to change our focus away from ELSA to Log Insight. The reasons being, High ingestion rate, promises of scalability, a very simple GUI yet highly customizable to our needs, dead simple to use, very low learning curve and pricing - Not buying an amount of data to ingest, but buying a per OSI license made the pricing very transparent and simple to scale. When version one of Log Insight was released we jump on it and started a POC. And the rest the say is history.

Environment is highend enterprise, based on Cisco nexus/ASA/PIX, HP Blades, F5 load balancers, VMware, HDS/IBM/EMC, Netbackup etc. 150 hosts, 3500 VMs running as an service provider/cloud service. Daily ingesting around 150GB of data.

When I compare Log Insight to Splunk which I consider the defacto standard for enterprise sysloging, Log Insight is much easier to use, to customize(extract fields) and the concept of content packs/widgets is dead easy to work with and adds instant value, which you don't get with any solution that I know of.

Today I work as an consultant on Log Insight and vRops mainly. To give you an example of how Log Insight can be used to troubleshoot - I had a customer call me on a Sunday. They had deploy three new VMware hosts and they would spontaneous reboot, no core dump, no warning, no nothing. The next day, Monday they were to in source IT operations from a large vendor. So of to a very bad start if they didn't get this fixed soon. As there were nothing to go on in vCenter, no alerts etc. I jumped into Log Insight and looked to see if there were anything in the logs on the hosts that rebooted, but no luck. I then turn a way from looking at the host and started to look generally at the environment, on if the best places to start is to view the content pack called vSphere and selecting the Problems dashboard - There I could see there were a lot of storage related issues and that they were all on the new hosts. Looking at the dashboards around storage/scsi in the same content pack, reviled a lot of severe SCSI errors FC frame failing CRC etc.

So I now knew were the problem was, so I compared the HBA of new and old hosts and found that the new host were running at a higher speed the all the old HBAs. Talking to the customer reviled at this was an know configuration issue, the customer changed the speed of the HBA and the problem was gone!

So Log Insight did give me the answer, but then no other solution would have been able to do so for this issue. But what it did was point me to the error in the environment and helped me quickly understand the problem. The solution will most like be different from case to case.

So content packs helps you a lot, and can help you be more proactive. Content pack exists for Cisco ASA/Nexus, Windows, Linux, some applications, like AD, tomcat, etc.

If there is not content pack for a solution, it doesn't mean Log Insight can be used, it just means that Log Insight works like every other syslog solution and just stores the data and the you will be able to create your own fields, widgets, dashboards, alarms etc. or don't if that's not your use case.

All of what you have described can be done with Log Insight.

Hope this helps

[zaspam]

I highly appreciate you opinion and I would like to say thanks for such an in-depth description, since I too will probably have to go through a similar evaluation scenario.

Thanks