vROPS sending log to LI using LI Agent

[fduranti]

I'm trying to use the solution for vrops logging to loginsight (Cloud Management Marketplace | Solution Exchange) and I'm having some problem as reported in this post of vrops forum... VROPS Automatically Change Log Insight configuration (in a wrong way)

There's anyone using those solutions togheter and had the same problem (vrops changing the li configuration automatically) ?

I had to exclude some logging because from my installation of vrops I was getting about 20 Million event/day only for vrops most of which are INFO/DEBUG.

Does anyone have some guideline on which configuration can be done?

there's seems to be only small documentation on log configuration... i saw some articles that talk about the log4j files but it seems that they got reset to default if I edit the configuration from the graphical gui too.

Regards

Francesco

[sflanders]

vR Ops remote syslog configuration from the UI and the Log Insight content pack for vR Ops are two completely separate things. If you goal is to use the Log Insight content then you should not being configuring remote syslog from the vR Ops UI and if you already have then you should remove all configuration you applied from the UI. The Log Insight content pack for vR Ops requires the Log Insight agent with a specific agent configuration. I am not sure what you mean about changing the log section impacts the liagent.ini file, can you provide the exact steps? If you are saying that configuring remote syslog from the vR Ops UI makes changes to the liagent.ini file this should not be the case because the UI I believe uses log4j for sending syslog and is not aware of the liagent today.

As for the large number of events you are receiving -- I have experienced the same then when I used the vR Ops remote syslog configuration from the UI and I changed the default logging level to something more verbose. Be sure to unconfigure everything from the UI. With the liagent I see about 50 EPS or 2M events a day for a single node instance. I suspect a large vR Ops instance and/or clustered instance would be even higher. In short, vR Ops is very verbose and it may make sense to reduce the severity level from the default.

[fduranti]

I'm trying to use the content pack but I've some problem with logging configuration. It also send too many info/debug/audit logs so if there's some suggesion on logs that should remain verbose and what can be set only to error or warning it would be nice... I see also that many of the warning or errors seems something that could not be solved and it seems also that logs are replicated (adapter logs are in the collector.log file and in the adapter log file).

Getting back to the problem of the syslog/loginsight configuration I was thinking the same about the configuration of syslog but probably with 6.0.1 it configure loginsight too so there's definitively something wrong.

I'll try to explain what happen with some screenshot....

This is my loginsight.ini configuration before any change (clean configuration that I put there manually.

I go th vrops (6.0.1+HF1) and open the configuration of a log... I've just disabled (time ago the output log flag.... the syslog hostname and post are something I've not put in, it seems to get them from the liagent.ini file.

Doing a change and saving the configuration this is what I get:

The liagent.ini file changed

I've done a test on a freshly installed test server I'm using for some test... I've got to the configuration and enabled syslog putting the server for loginsight and port 9000 on the configuration for collection... it also added the collector section to the liagent.ini file:

# Client-side configuration of VMware Log Insight Agent

# See liagent-effective.ini for the actual configuration used by VMware Log Insight Agent

[server]

proto = syslog

port = 9000

reconnect = 30

hostname = itromvlog01.q8int.com

# Time in minutes to force reconnection to the server

# If omitted the default value is 30

#reconnect=30

[storage]

max_disk_buffer = 200

#max_disk_buffer - max disk usage limit (data + logs) in MB:

# 100 - 2000 MB, default 200

#max_disk_buffer=200

[logging]

debug_level = 0

[filelog|COLLECTOR-collector]

include = collector*.log

tags = {"appname":"vROps", "logtype":"COLLECTOR"}

directory = /usr/lib/vmware-vcops/user/log

I think LI team and vRops team should talk a bit about the LI/vRops integration configuration at this point...

[sflanders]

I see the issue -- you are editing the liagent-effective.ini file. For whatever reason, the header in that file is not correct. It should state that you should not edit that file as changes will be overridden. The liagent-effective is a combination of what is in liagent.ini (client-side) and what is under /admin/agents on LI (server-side). You changing syslog configuration through the vR Ops UI has no impact on the agent configuration, but the agent is checking the client-side and server-side configurations and updating the effective configuration. Based on your screenshots, you have port 514 defined in liagent.ini and thus the change. Long story short, do not edit the effective file, edit the liagent.ini file and you will be good.

[fduranti]

HI, I'm sure I'm editing the correct file. The file liagent.ini change if I edit that configuration of vrops log setting.

I edit liagent.ini, save a configuration on vrops and i see the liagent.ini that chane and change only some parameters. it the get disconnnected from,liserver until i manually change the configuration back to the correct one.  I'm sure it's not the liagent-effective because I have some config on server that get pushed too. 

[fduranti]

Just to be clear (there's the cat command to display logs) I've redone some screenshot with a default manually copied from another server liagent.ini and the thing I get after pushing ok on the editing of log inside vrops 6.0.1. vrops change the loginsight configuration.

[sflanders · Accepted solution]

Confirmed -- sorry for the back and forth on this. Testing on my end was not done with 6.0.1. So looks like you cannot use content pack and UI remote syslog with 6.0.1. Thanks for reporting -- I will get this tracked down with the vR Ops team. Did you open a support case for this? If not, could you and ensure it is assigned to vR Ops?

[fduranti]

I've opened now a SR pointing to this forum post.

SR number 15645320804

[dsohayda]

What exactly doesn't work? Is there a compatibility issue with LI2.5 and vrops 6.0.1?

I'm trying to get the vROps 6.x content pack working with our vrops 6.0.1 install, but the dashboard in LI isn't showing any data. The log files on my vrops node show  "CurlConnection::Config transport error: Couldn't connect to server" errors in the loginsight-agent directory too.

I updated the liagent.ini file and restarted the log insight agent on the vrops node, but after a few hours nothing seems to be collecting.

I used this blog article to help with the parameters that need to be changed on the ini file.

vRealize Operations Manager Content Pack for Log Insight - VMware Blogs

Another strange thing I noticed; If I try to disable the remote syslog option from the vrops UI I click ok, the window changes to a saved successfully message, but then if I go back into that setting it still shows as being enabled. Not sure what is going on with this.

[sflanders]

I documented the exact issue here: Log Insight Management Pack for vRealize Operations Manager - SFlanders.net. I hope it helps!

[dsohayda]

Did you mean to link to your other article? vRealize Operations Manager Content Pack for Log Insight - SFlanders.net

I'm not clear on what you are saying needs to be done though. What do you mean by the /admin/agents page server side? I think you're saying it's best practice to manage the agent from the vrops UI and not the ini file, but that's the opposite of what the documentation says. In the article you linked to you mention the configuration portion of the log insight agent in the vrops UI is grayed out. I upgraded the log insight management pack in vrops using the pak file on solutions exchange from 6.0.2244391 to 6.0.2338756 and I can now launch in context from vrops to LI, but still no data in LI regarding my vrops node.

Thanks

[sflanders]

I did yes the point is that you should NOT configure syslog export from the vR Ops UI if running 6.0.1 or newer. The blog posts also suggests using the /admin/agents page as that is a best practice for all agent configuration -- the documentation recommends client-side because configuration is all or nothing in LI 2.5 (i.e. applies to all agents or no agents) and since the agent configuration needs to be different per vR Ops node this does not work for multi-node or multi-instance vR Ops configurations today. See the Tech Preview program on https://loginsight.vmware.com for ways around this.

Anyways -- in regards to your issue, I would suggest logging in the agent log file as it will report any errors in your configuration. It can be found in /var/log/loginsight-agent. Searching for "error" should help narrow it down.

[sflanders]

Side note -- happy to assist with your issue, but it appears to be different from this thread so please start a new one to discuss your issue. Thanks!