LogFile Handling – Event_Marker

[vMarkusK1985]

Hello,

I`m struggling with a special Log Format:

[Critical] From: CMA@xx02xxx0001.xxx.loc "CC02D2D0001_CC01XXX_Agent_GW1 [GW 2768:1:4783761995814005136]"  Time: 24.06.2015 09:46:32

[90:17]   Cannot start NETIO process ([0] The operation completed successfully. ) => aborting

[Critical] From: CMA@xx02xxx0001.xxx.loc "CC02D2D0001_CC01XXX_Agent_GW1 [GW 2768:2:4783761995814005136]"  Time: 24.06.2015 09:46:33

[90:17]   Cannot start NETIO process ([0] The operation completed successfully. ) => aborting

Or:

Report Messages Other Than Normal [xx01xxx0002.xxx.loc]:

[Critical] From: CSM@xx01xxx0002.xxx.loc "oc_CC01-Xxxx_DailyAgent_ORAARCH2"  Time: 24.06.2015 09:45:37

  Invalid device type used for replication!

[Critical] From: CSM@xx01xxx0002.xxx.loc "oc_CC01-Xxxx_DailyAgent_ORAARCH2"  Time: 24.06.2015 09:45:37

  Invalid device type used for replication!

How can I handle this with a RegEx?

[sflanders]

Looking at those formats I would use:

event_marker=^\[\w+\] From:

I hope this helps!

[vMarkusK1985]

Hm,

nothing is processed with: event_marker=^\[\w+\] From:

[sflanders]

Something else is wrong with your configuration then as I just tested on 2.5-ga and the latest TP and both work Here is the sample I tried with:

[filelog|test]

directory=/var/log/test

event_marker=^\[\w+\] From:

Attached is a screenshot with and without the event_marker. I hope this helps!

[sflanders]

I should have added -- check the agent log file and search for 'error' or 'warng' as that should tell you where your configuration issue is.

[vMarkusK1985]

No Config Error Reported:

2015-06-25 14:44:34.765331 0x00001600 <trace> Config:116        | Reading configuration from: C:\ProgramData\VMware\Log Insight Agent\liagent.ini

2015-06-25 14:44:34.765331 0x00001600 <trace> Config:133        | Reading configuration received from server. Hash = d41d8cd98f00b204e9800998ecf8427e

2015-06-25 14:44:34.765331 0x00001600 <trace> Config:88          | The current effective configuration is dumped into file C:\ProgramData\VMware\Log Insight Agent\liagent-effective.ini

2015-06-25 14:44:34.765331 0x00001600 <trace> AgentDaemon:273    | AgentDaemon reconfiguring...

The Config:

[winlog|Application]

channel=Application

[winlog|Security]

channel=Security

[winlog|System]

channel=System

[filelog|HPDPSync]

directory= D:\Replication\log

tags = {"appname":"HPDPSync"}

exclude_fields=hostname

[filelog|HPDPSessionError]

directory= D:\OmniBackReport\

include= errors.log

tags = {"appname":"HPDPSessionError"}

event_marker= ^\[\w+\] From:

There Is nothing collected from this section: [filelog|HPDPSessionError]. When I disable the "event_marker= ^\[\w+\] From:" line the whole Log is precessed...

[sflanders]

OK, I just tested this on the Windows agent and it worked as well. Next questions:

* What exact version of the agent are you running?

* Can you try to send test messages? For example, add a new section:

[filelog|Test]

directory= D:\OmniBackReport\

include= test.log

tags = {"appname":"HPDPSessionError"}

event_marker= ^\[\w+\] From:

Then copy the error.log file to test.log -- note if error.log is really big (more than a couple MBs) then I would suggest just copying the two example lines in your first post and pasting them into a file called test.log, saving the file and close the text editor

[vMarkusK1985]

Appliance Version: 2.5 GA (2.5.0-2347850)

Agent Version: 2.5.0.2347850

I added the Test Lines and created a copy of the File. Its thew same result, no data is processed in vLI...

[vMarkusK1985]

Hi sflanders,

Have you found the time to check the behavior?

[sflanders]

Hey sorry -- things have been very busy for me. I just deployed LI 2.5 GA with the 2.5 GA agent on a Windows 8 box. I copied your configuration and used your sample logs -- I get results in LI and the event_marker works perfectly. The only thing I can think of is that you may have copied/pasted my response above and it has some special characters in it. Please be sure to manually type out the configuration and try again. It should work. Sorry again for the delay.

[vMarkusK1985]

Hey,

Sorry for my late response...

Today I had time for further testing.

I don’t think it’s a C&P Problem.

I successfully Processed the log without the event_marker:

Then tried to do a "Search":

-> Search with string or RegEx has no result.

That’s really curious!

[vMarkusK1985 · Accepted solution]

[filelog|HPDPSessionError]

directory= D:\OmniBackReport\

include= errors.log

tags = {"appname":"HPDPSessionError"}

charset=UTF-16LE

event_marker=^\[\w+\] From: