Log Insights - Agents - Parser RegExp

[RNaray308]

Hi,

I am a newbie to LOG INSIGHTS and did good in figuring out with the help of this community and online documentations. However, I am stuck with Regexp and because of that I am not able to retrieve the data as expected in the tool.

I have a very large log file on one of the server with information close to below.

Issue:- In the Interactive Analytics it is displaying one row for each line, hence the below log is shown as 12 different lines for each timestamp.

[2020-06-10T15:28:10-04:00] [OBIPS] [TRACE:1] [] [saw.rpc.server.processHeartbeat] [ecid: xxxxxxxxxxxxxxxxxxxxxx,0] [tid: 3793676032] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Processing heartbeat message.[[

File:socketrpcserver.cpp

Line:399

Location:

saw.rpc.server.processHeartbeat

saw.rpc.server.responder

saw.rpc.server

saw.rpc.server.handleConnection

saw.rpc.server.dispatch

saw.threadpool.socketrpcserver

saw.threads

]]

So I have used the following RegExp code which works perfect when I checked in https://regex101.com/​ . But it haven't improved my results.

(?<DateTime>\[.*?\]) (?<OBIPS>\[.*?\]) (?<TRACE>\[.*?\]) (?<EMP>\[.*?\]) (?<FRISTMSG>\[.*?\]) (?<ECID>\[.*?\]) (?<TID>\[.*?\]) (?<SINAME>\[.*?\]) (?<IDDNAME>\[.*?\]) (?<IDDGUID>\[.*?\]) (?<USRID>\[.*?\[)(?<COMPLETE>\[\s*[\w\W]*?\]\])

I have attached some screenshots and the above scenario in the document. Could you please take a look at it and help me out?

I greatly appreciate your responses and help.

Thank you,

Rama

[Cederberg]

Hi.

To get all the text in to one logevent you need to define a event marker in your filelog config it's right under the directory line in the gui. I see in your screenchot you only had the default ^ in the textbox for event marker and ^ means start of line if i'm not misstaken. So you need to do a regexp for the start of every event. If for example every line starts with the timecode you have in the example you gave you need to match that in event marker. ^\[\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}-\d{2}:\d{2}\] will match lines that starts like this [2020-06-10T15:28:10-04:00] and put everything after that into one log event until the next match.

The parser can then be used to parse the event into fields for filtering.

Hope that helps

Best regards

//Mattias