Microsoft - Windows Content pack .. problem with i...

HarryWer · ‎01-08-2015

I have installed Log Insight 2.6 GA and have tried to ingest an application log file using these parameters on the target server;

; Dynamic file representing the effective configuration of VMware Log Insight Agent (merged server-side and client-side configuration)

; DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

; Creation time: 2015-01-08T16:11:31.137877

[server]

hostname=qloginsight

[winlog|Application]

channel=Application

[winlog|Security]

channel=Security

enabled=no

[winlog|System]

channel=System

[filelog|AAPFormGuide]

directory=D:\TABLogs\RWWA.AapFormGuide.Host

include=*.log

tags={"appname":"AAPFormGuide"}

channel=TABLogs

[filelog|PhoenixPush]

directory=D:\TABLogs\PhoenixPush

include=*.log

exclude=*.AuditTrace.*;*.ServiceTrace.*

tags={"appname":"PhoenixPush"}

channel=TABLogs

I can see the relevant Windows event log entries but no application log details for the machine.

An example of entries in the application log in "D:\TABLogs\RWWA.AapFormGuide.Host" for file "20150106.01.AapFormGuide.RAWDMWS104V.log" ;

############################################

LOG STARTED 2014-12-16 11:56:01,750

############################################

2015-01-06 00:00:44,899 [9] INFO AAP1000O MissingSilksProcessor - About to update silks 6/01/2015 00:00:44.

2015-01-06 00:00:44,899 [9] INFO AAP1000O RepositoryLocker - Requesting lock for AAP_MissingAapSilksCheck.

2015-01-06 00:00:44,899 [9] INFO AAP1000O RepositoryLocker - Lock succeeded for AAP_MissingAapSilksCheck.

2015-01-06 00:00:44,899 [9] INFO AAP1000O MissingSilksProcessor - Found 0 acceptor silks to update. Elapsed time 4 ms.

2015-01-06 00:00:44,899 [9] INFO AAP1000O MissingSilksProcessor - Update silks completed. Elapsed time 4 ms.

2015-01-06 00:00:44,899 [9] INFO AAP1000O RepositoryLocker - Releasing lock for AAP_MissingAapSilksCheck.

2015-01-06 00:00:44,899 [9] INFO AAP1000O RepositoryLocker - Lock released successfully for AAP_MissingAapSilksCheck.

2015-01-06 00:00:44,899 [9] INFO AAP1000O RepositoryLocker - Requesting lock for AAP_MissingRiseSilksCheck.

2015-01-06 00:00:44,915 [9] INFO AAP1000O RepositoryLocker - Lock succeeded for AAP_MissingRiseSilksCheck.

2015-01-06 00:00:44,915 [9] INFO AAP1000O RepositoryLocker - Releasing lock for AAP_MissingRiseSilksCheck.

2015-01-06 00:00:44,915 [9] INFO AAP1000O RepositoryLocker - Lock released successfully for AAP_MissingRiseSilksCheck.

2015-01-06 00:01:46,368 [9] INFO AAP1000O FileProcessor - Checking for form guides in FTP location: InFormGuideDev

2015-01-06 00:01:47,337 [9] ERROR AAP1001D FileProcessor - Exception in FormGuide FileProcessor

System.Net.WebException: Unable to connect to the remote server

at System.Net.FtpWebRequest.GetResponse()

at RWWA.AAPFormGuide.ServiceImpl.FtpProcessing.FtpProcessor.GetFileList()

at RWWA.AAPFormGuide.ServiceImpl.FileProcessor.GetFtpFileList()

at RWWA.AAPFormGuide.ServiceImpl.FileProcessor.Process()

at RWWA.AAPFormGuide.ServiceImpl.FileProcessor.ProcessInComingFiles()

Any ideas ?

Regards,

Harry Werkman

sflanders · ‎01-09-2015

Hey Harry,

Only winlog sections contain a channel= configuration option. I suspect if you look in the agent log directory you will see an error where the filelog sections are being ignored because of an invalid configuration. If you remove the channel options from each filelog section I suspect your issue will be resolved (be sure to check the log file after to confirm no additional errors). I hope this helps!

Hope this helps! === If you find this information useful, please award points for "correct" or "helpful". ===

HarryWer · ‎01-12-2015

This is my effective liagent-effective.ini

; Dynamic file representing the effective configuration of VMware Log Insight Agent (merged server-side and client-side configuration)

; DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN

; Creation time: 2015-01-13T14:37:21.399581

[server]

hostname=qloginsight

[winlog|Application]

channel=Application

[winlog|Security]

channel=Security

enabled=no

[winlog|System]

channel=System

[filelog|AAPFormGuide]

directory=D:\TABLogs\RWWA.AapFormGuide.Host\

event_marker=^[^\s]

[filelog|PhoenixPush]

directory=D:\TABLogs\PhoenixPush\

event_marker=^[^\s]

... and still no result unfortunately.

This is the output into the liagent_2015_01_13_16.log file

2015-01-13 14:37:21.383956 0x00000158 <trace> AgentDaemon:90	\| AgentDaemon start requested.
2015-01-13 14:37:21.383956 0x00000ba0 <trace> WinService:250	\| WinService::SetServiceStatus START_PENDING, Win32ExitCode = 0

2015-01-13 14:37:21.383956 0x00000158 <trace>

	Agent Build	: 2.5.0.2347850
	Start Time	: 2015-01-13 14:37:21.383956
	Running as user : RWWAQ\SYSTEM
	Our Process ID : 3488
	Executable Path : C:\Program Files (x86)\VMware\Log Insight Agent\liwinsvc.exe
	Operating System: Microsoft Windows Server 2008 R2 Standard 6.1.7601 Service Pack 1 64-bit

2015-01-13 14:37:21.383956 0x00000158 <trace> AgentDaemon:104	\| Data directory: "C:\ProgramData\VMware\Log Insight Agent"
2015-01-13 14:37:21.383956 0x00000158 <trace> AgentDaemon:108	\| Initializing storage...
2015-01-13 14:37:21.383956 0x00000158 <trace> DbConnection:34	\| Opening database file C:\ProgramData\VMware\Log Insight Agent\storage\liagent.db
2015-01-13 14:37:21.383956 0x00000158 <trace> DbConnection:51	\| Database "C:\ProgramData\VMware\Log Insight Agent\storage\liagent.db" opened successfully
2015-01-13 14:37:21.383956 0x00000158 <trace> DbStorage:220	\| Checking database integrity...
2015-01-13 14:37:21.383956 0x00000158 <trace> DbStorage:258	\| Database integrity check done.
2015-01-13 14:37:21.383956 0x00000f28 <trace> Logger:136	\| Thread "DbStorage Maintenance" has id 0x00000f28
2015-01-13 14:37:21.383956 0x00000158 <trace> EventQueue:32	\| EventQueue::EventQueue stored event id's: min = 0, max = 0
2015-01-13 14:37:21.383956 0x00000f28 <trace> DbStorage:454	\| DbStorage maintenance thread started.
2015-01-13 14:37:21.399581 0x00000158 <trace> AgentDaemon:114	\| Agent UID:37D41A42-12B8-59E1-492D-8314B4A4E732
2015-01-13 14:37:21.399581 0x00000158 <trace> Config:116	\| Reading configuration from: C:\ProgramData\VMware\Log Insight Agent\liagent.ini
2015-01-13 14:37:21.399581 0x00000d34 <trace> Logger:136	\| Thread "DirectoryMonitor" has id 0x00000d34
2015-01-13 14:37:21.399581 0x00000158 <trace> Config:133	\| Reading configuration received from server. Hash = 830bf663d9b110feaebc1b6d3908e45c
2015-01-13 14:37:21.399581 0x00000158 <warng> Config:320	\| Config key [server].hostname received from server will be ignored.
2015-01-13 14:37:21.399581 0x00000158 <trace> Config:88	\| The current effective configuration is dumped into file C:\ProgramData\VMware\Log Insight Agent\liagent-effective.ini

2015-01-13 14:37:21.399581 0x00000158 <trace> DbConnection:145 | Setting SQLite cache_size = 2868224 bytes

2015-01-13 14:37:21.399581 0x00000158 <trace> Config:251	\| Configuration key server.proto is not specified. Using default: cfapi
2015-01-13 14:37:21.399581 0x00000158 <trace> AgentDaemon:201	\| Creating cfapi transport
2015-01-13 14:37:21.399581 0x00000158 <trace> Config:242	\| Read config param server.hostname = qloginsight
2015-01-13 14:37:21.399581 0x00000158 <trace> Config:291	\| Configuration key server.ssl is not specified. Using default: no

2015-01-13 14:37:21.399581 0x00000158 <warng> CFApiTransport:123 | Config param server.port is not specified. Using default: 9000

2015-01-13 14:37:21.399581 0x00000158 <trace> Config:222

| Configuration key server.reconnect is not specified. Using default: 30

2015-01-13 14:37:21.415206 0x00000158 <trace>

	Agent Up Time 00:00:00.033000
	Observed Events : 0 (total events seen for all log sources since Agent started or changed server)
	Collected Events: 0 (=Observed-Dropped)
	Sent Events	: 0 (delivered to destination server)
	Dropped Events : 0 (dropped due to local storage overflow or rejected by the server)
	Sending Rate	: 0.00 EPS (average for last minute)
	DB File Size	: 28,672 bytes
	CPU Usage	: 0 % (average for last 0 seconds)
	Connection	: cfapi://qloginsight:9000
	Hostname (FQDN) : RAWRVMC104V.rwwaq.com.au
	Disk Space Used : 4,000,197 bytes
	Machine UID	: 37D41A42-12B8-59E1-492D-8314B4A4E732
	Agent UID	: 37D41A42-12B8-59E1-492D-8314B4A4E732

	Performance Counters ------------------------------ For Last 0 seconds ------------------------	-------------------------- Cumulative ---------------------------
	count	min(us)	max(us)	avg(us)	total(us)	count	min(us)	max(us)	avg(us)	total(us)

Internal Debug Counters -------------------------------------------------------------------------	-----------------------------------------------------------------
DbStorage::CheckDb	1	324	324	324	324	1	324	324	324	324
DbStorage::InitDb	1	1,291	1,291	1,291	1,291	1	1,291	1,291	1,291	1,291
-------------------------------------------------------------------------------------------------	-----------------------------------------------------------------

2015-01-13 14:37:21.430831 0x00000158 <trace> AgentDaemon:137

| Starting collectors...

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:28 | ConfigureAndStart invoked for collector: winlog

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:53 | Configuring winlog

2015-01-13 14:37:21.430831 0x00000158 <trace> Config:280

| Read config param winlog|Security.enabled = no

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:55 | Configuration of winlog is done

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:62 | Starting winlog

2015-01-13 14:37:21.430831 0x00000158 <trace> WinLogSession:407 | Subscribed to channel <Application>

2015-01-13 14:37:21.430831 0x00000158 <trace> WinLogSession:407 | Subscribed to channel <System>

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:65 | Started winlog

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:28 | ConfigureAndStart invoked for collector: filelog

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:53 | Configuring filelog

2015-01-13 14:37:21.430831 0x00000cd8 <trace> Logger:136

| Thread "WinLogMonitor" has id 0x00000cd8

2015-01-13 14:37:21.430831 0x00000cd8 <trace> WinLogCollector:320| WinLogMonitor thread begin

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:55 | Configuration of filelog is done

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:62 | Starting filelog

2015-01-13 14:37:21.430831 0x00000158 <trace> FLogCollector:246 | Subscribed to channel <AAPFormGuide>.

2015-01-13 14:37:21.430831 0x00000158 <trace> FLogCollector:246 | Subscribed to channel <PhoenixPush>.

2015-01-13 14:37:21.430831 0x00000500 <trace> Logger:136	\| Thread "DirectoryMonitor" has id 0x00000500
2015-01-13 14:37:21.430831 0x00000bb0 <trace> Logger:136	\| Thread "DirectoryMonitor Polling" has id 0x00000bb0
2015-01-13 14:37:21.430831 0x00000eac <trace> Logger:136	\| Thread "FLogThreadPool" has id 0x00000eac
2015-01-13 14:37:21.430831 0x00000ebc <trace> Logger:136	\| Thread "FLogThreadPool" has id 0x00000ebc

2015-01-13 14:37:21.430831 0x00000158 <trace> EventCollector:65 | Started filelog

2015-01-13 14:37:21.430831 0x00000ee8 <trace> Logger:136	\| Thread "FLogThreadPool" has id 0x00000ee8
2015-01-13 14:37:21.430831 0x00000158 <trace> AgentDaemon:142	\| Collectors started. Starting transport...
2015-01-13 14:37:21.430831 0x00000830 <trace> Logger:136	\| Thread "FLogThreadPool" has id 0x00000830
2015-01-13 14:37:21.430831 0x00000ac4 <trace> Logger:136	\| Thread "CFApiTransport" has id 0x00000ac4
2015-01-13 14:37:21.430831 0x00000158 <trace> AgentDaemon:148	\| AgentDaemon started successfully

2015-01-13 14:37:21.430831 0x00000ac4 <trace> CFApiTransport:350 | Connecting to server qloginsight:9000

2015-01-13 14:37:21.430831 0x00000f44 <trace> Logger:136	\| Thread "AgentDaemon Reconfiguration" has id 0x00000f44
2015-01-13 14:37:21.430831 0x00000f44 <trace> AgentDaemon:247	\| Reconfiguration thread started
2015-01-13 14:37:21.430831 0x00000158 <trace> WinService:250	\| WinService::SetServiceStatus RUNNING, Win32ExitCode = 0

2015-01-13 14:37:21.446456 0x00000ac4 <trace> CFApiTransport:367 | Connection successfully established

HarryWer · ‎01-13-2015

Success ...

This is the .ini file I used ...

; Client-side configuration of VMware Log Insight Agent.

; See liagent-effective.ini for the actual configuration used by VMware Log Insight Agent.

[server]

; Log Insight server hostname or ip address

; If omitted the default value is LOGINSIGHT

hostname=qloginsight

; Set protocol to use:

; cfapi - Log Insight REST API

; syslog - Syslog protocol

; If omitted the default value is cfapi

;

;proto=cfapi

; Log Insight server port to connect to. If omitted the default value is:

; for syslog: 514

; for cfapi without ssl: 9000

; for cfapi with ssl: 9543

;port=9000

;ssl - enable/disable SSL. Applies to cfapi protocol only.

; Possible values are yes or no. If omitted the default value is no.

;ssl=no

; Time in minutes to force reconnection to the server

; If omitted the default value is 30

;reconnect=30

[storage]

;max_disk_buffer - max disk usage limit (data + logs) in MB:

; 100 - 2000 MB, default 200

;max_disk_buffer=200

[logging]

;debug_level - the level of debug messages to enable:

; 0 - no debug messages

; 1 - trace essential debug messages

; 2 - verbose debug messages (will have negative impact on performace)

;debug_level=0

[winlog|Application]

channel=Application

[winlog|Security]

channel=Security

[winlog|System]

channel=System

[filelog|PhoenixPush]

directory=D:\TABLogs\PhoenixPush

include=*.*.PhoenixPush.*.log

event_marker=^[^\s]

[filelog|AAPFormGuide]

directory=D:\TABLogs\RWWA.AapFormGuide.Host

event_marker=^[^\s]

Thanks to all who helped in resolving this problem.

Harry W.

sflanders · ‎01-13-2015

Excellent, glad you were able to figure it out! Can you please mark your question as answered? If you do not see the option then you need to try a different browser.

Hope this helps! === If you find this information useful, please award points for "correct" or "helpful". ===

All

Microsoft - Windows Content pack .. problem with injesting a application log file

Content Packs