[Winlogbeat] Can't read from continously updated .evtx file.

[theodiefenthal]

I have winlogbeat running on a system where multiple hosts forward their events to a single host, which in turn runs winlogbeat to put all of those events to kafka. I have a load of ~5000 events per second, but winlogbeat manages only 2000 events per second on my machine. I just wanted to switch to reading .evtx files directly in hope that this will work out significantly faster.

In my windows event logging, I setup a max file size of 100GB and that new events overwrite old ones in that file.

My config looks like this:

winlogbeat.event_logs:
  - name: 'E:\Win_logs\CollectedEvents.evtx'
    forwarded: true
    batch_read_size: 512

On the first attempt, I got this error:

2020-05-25T11:48:07.286+0200    WARN    eventlog/wineventlog.go:218    WinEventLog[E:\Win_logs\CollectedEvents.evtx] Failed to seek to bookmarked location in E:\Win_logs\CollectedEvents.evtx (error: The parameter is incorrect.). Recovering by reading the log from the beginning. (Did the file change since it was last read?)
2020-05-25T11:48:07.286+0200    WARN    beater/eventlogger.go:138    EventLog[E:\Win_logs\CollectedEvents.evtx] Read() error: failed to recover from RPC_S_INVALID_BOUND: failed to seek to beginning of log: The parameter is incorrect.
2020-05-25T11:48:07.286+0200    INFO    beater/eventlogger.go:113    EventLog[E:\Win_logs\CollectedEvents.evtx] Stop processing.
2020-05-25T11:48:07.286+0200    WARN    beater/eventlogger.go:116    EventLog[E:\Win_logs\CollectedEvents.evtx] Close() error. The handle is invalid.

On the next start, I received this one:

2020-05-25T11:55:48.176+0200    WARN    eventlog/wineventlog.go:218    WinEventLog[E:\Win_logs\CollectedEvents.evtx] Failed to seek to bookmarked location inE:\Win_logs\CollectedEvents.evtx (error: The parameter is incorrect.). Recovering by reading the log from the beginning. (Did the file change since it was last read?)
2020-05-25T11:55:48.176+0200    WARN    beater/eventlogger.go:108    EventLog[E:\Win_logs\CollectedEvents.evtx] Open() error. No events will be read from this source. failed to seek to beginning of log: The parameter is incorrect

I'm currently using WinLogBeat 7.2, probably switching to 7.6.2 soon. Due to #18640 , I can't test the latest 7.7 version.

Note that this 100GB is already full of events. I can follow that old events are already discarded. Might this cause a problem?

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[andrewkroh]

I just wanted to switch to reading .evtx files directly in hope that this will work out significantly faster.

The evtx reader uses the Windows API to read the data. It is meant for reading archived evtx files, not active logs. It will have about the same read rate since it uses the Windows API.

[andrewkroh]

Note that this 100GB is already full of events. I can follow that old events are already discarded. Might this cause a problem?

If events are discarded before they can be read by Winlogbeat, yes, that would be a problem since you'd lose events. Using the large log size affords you some time for the reader to catch up, but if the high log rates are sustained for too long then the reader may not catch up before rotation occurs.

[andrewkroh]

Regarding throughput, in 7.7 Winlgobeat added an experimental event log reader that tries to address the read-throughput limits. See https://www.elastic.co/guide/en/beats/winlogbeat/7.7/configuration-winlogbeat-options.html#_event_logs_api. If you ever do get to a point where you can test the latest 7.7 then give that a try and let us know what issues you encounter so we can improve it.

[botelastic]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[peas100]

Hi,
I have installed winlogbeat 7.11. This functionality of reading logs in continuous manner, is it supported now?
If not, is there a way to achieve this? (e.g. a config flag to set poll etl file every 30s).
I tried to consume channel directly but that only has Analytic channel i.e. the log output is in etl format.
Winlogbeat says it can't read/subscribe from direct channel.

[andrewkroh]

No, the feature is meant to read from archived files. Actively written files should be referenced in the config via the associated event log channel name.

[peas100]

Ok, thanks for your reply. Thing is, the Eventlog provider's log is available through EventTracingSession only, which writes logs to etl file continuously (once tracing is enabled). For e.g. Microsoft-Windows-Kernel-EventTracing. That's why I had to use this feature of reading events from file.
Can you share any pointers as to how to address this? Filebeat is able to read file but not able to parse. As per doc, I think winlogbeat was meant for parsing windows events OOB.

[andrewkroh]

Winlogbeat does not have support for Event Tracing (ETW) and the event trace log (ETL) files it produces. That's a separate Windows API from the Windows Event Log API. You may be able to read an ETL file that's been archived (not active) but it's not a supported or tested feature. The feature request for ETW is in #2073.