Log Insight CSV Parser problem

[focusdriver]

I added a csv parser to liagent.ini with fields and timestamp parser for an nlog-based text logfile.

The log format is ${longdate}\t${event-type:item="category"}\t{$level:uppercase=true}\t${message}\t${exception} as csv format in nlog.config.

I then added fields Date, categoty, level, message and exception to a parser definition in liagent.ini.

Also the date fields is parsed by a timestamp parser.

So how can I access the configured fields in the file log definition on the agent configuration in the appliance and use them in queries for dashboards?

[vmovsisyan]

more details needed: could you paste relevant filelog & parser sections?

[admin]

For csv parser the field list in the parser has to exactly match in number of fields and order of fields with the actual log. Please verify this.

This is an example from Microsoft sharepoint content pack:

[filelog|sp2010_sp_requestUsage]

; IMPORTANT: Change the directory as per the environment

directory=C:\ProgramData\VMware\Log Insight Agent\sharePoint\scripts\logs

include=SPRequestUsage.csv

event_marker=^\"SPRequestUsage\"

parser=sp2010_sp_requestUsage_parser

tags={"ms_product":"sharepoint"}

[parser|sp2010_sp_requestUsage_parser]

base_parser=csv

delimiter=","

fields=,ms_sp_pagereq_rowid,,,,,,,ms_sp_pagereq_weburl

Note: In the above example some fields are skipped. After the log is parsed using the above config ; user can query upon fields ms_sp_pagereq_rowid & ms_sp_pagereq_weburl in dashboards and interactive analytics.

Hope this helps.

[focusdriver]

Here is the snippet from the liagent.ini:

[parser|execnlogparser]
base_parser=csv
delimiter="\t"
fields=Timestamp,Category,Level,Message,Exception
field_decoder={"timestamp" : "tsp_parser"}
[parser|tsp_parser]
base_parser=timestamp
format=%Y-%m-%d %H:%M:%S
[parser|cfglogparser]
base_parser=csv
delimiter="\t"
fields=Timestamp,Level,Message,Exception
field_decoder= { "Timestamp" : "tsp_parser" }
[filelog|execlogfiles]
directory=c:\brmservice\logs
include=ExecutionService*.log
parser=execnlogparser
tags={"_logType":"brmexeclog"}
[filelog|cfglogfiles]
directory=c:\brmservice\logs
include=configurationservicelog.txt
parser=cfglogparser
tags={"_logType":"brmcfglog"}

And here is the NLog configuration:

<layout xsi:type="CSVLayout" delimiter="Tab">
      <column name="Date" layout="${longdate}" />
      <column name="Level" layout="${level:uppecase=true}"/>
      <column name="Message" layout="${message}" />
      <column name="Exception" layout="${exception:format=type,message,method;maxInnerExceptionLevel=5;innerFormat=shortType,message,method}" />
    </layout>

[focusdriver]

Ok the problem got solved after reading debug logging of the liagent log files.