Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement input buffering #82

Merged
merged 6 commits into from
Jun 9, 2021
Merged

Implement input buffering #82

merged 6 commits into from
Jun 9, 2021

Conversation

satta
Copy link
Member

@satta satta commented Jun 4, 2021

This PR improves the socket input in FEVER by introducing a buffer for the input channel after parsing EVE from the socket.
The buffer is implemented using a variable-size buffered Go channel (--in-buffer) which will ensure that FEVER can keep up reading from the socket even if the processing pipeline or the forwarding receiver temporarily blocks.
If the buffer fills completely, we will optionally (--in-buffer-drop) rather drop parsed events from the buffer than cause writes to the input socket to block. This is all done to prevent Suricata from ever having to block its internal pipelines, leading to packet drops with an even wider impact.

We also send new metrics if the socket input is selected: input_buffer_length and input_buffer_dropped.

@satta satta requested a review from 0mbi June 4, 2021 15:12
@satta satta added the enhancement New feature or request label Jun 6, 2021
README.md Outdated Show resolved Hide resolved
cmd/fever/cmds/run.go Outdated Show resolved Hide resolved
input/input_socket.go Show resolved Hide resolved
@0mbi 0mbi merged commit 67ffdba into DCSO:master Jun 9, 2021
@satta satta deleted the in-buffering branch June 9, 2021 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants