-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
decode: several fixes #98
Merged
yaauie
merged 5 commits into
logstash-plugins:main
from
yaauie:handle-embedded-newlines-and-carriage-returns
Oct 26, 2022
Merged
decode: several fixes #98
yaauie
merged 5 commits into
logstash-plugins:main
from
yaauie:handle-embedded-newlines-and-carriage-returns
Oct 26, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Per CEF spec v25 (2017-09-28): > *Multi-line* fields can be sent by `CEF` by encoding the newline character > as `\n` or `\r`. Note that multiple lines are only allowed in the value part > of the extensions While this plugin has long _encoded_ multiline extension values and other escape sequences, our _decode_ has only supported escaped backslashes or escaped equals signs, and with this change becomes compliant with this portion of the spec. Note that due to the preexisting newline-centric normalization, round-trip encode/decode cycle is only _semantically_ guaranteed.
339e6cd
to
c4555ee
Compare
ff9bbf2
to
80d0a07
Compare
When encountering malformed-CEF or non-CEF payloads, this plugin now emits helpful descriptive log messages, and prevents data-loss and corruption by emitting an event tagged with `_cefparsefailure` containing the bytes it received. This set of changes catches 3 distinct cases of malformed payloads. - missing one or more of the 7 required CEF header fields; a payload that does not have all 7 unescaped-pipe-terminated header fields cannot be reliably interpreted as CEF (prevents corruption). - containing something OTHER than a sequence of key=value pairs in the extensions space (prevent data-loss; previously when extensions were invalid they were silently omitted) - containing unescaped newlines (prevents corruption; previously data after the first newline was injected into the currently-parsed extension field) In catching these classes of malformed inputs, this changeset also resolves logstash-plugins#99 in which our failure to detect a malformed input proactively caused an unhelpful `NoMethodError` message to be logged before a `_cefparsefailure`-tagged event was emitted.
80d0a07
to
2f0c75e
Compare
mashhurs
approved these changes
Oct 26, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changeset includes three fixes related to decoding:
CR/LF in Extension Values
Per CEF spec v25 (2017-09-28):
While this plugin has long encoded multiline extension values and other
escape sequences, our decode has only supported escaped backslashes or
escaped equals signs, and with this change becomes compliant with this portion
of the spec. Note that due to the preexisting newline-centric normalization,
round-trip encode/decode cycle is only semantically guaranteed.
Intercepting Invalid Payloads
Buffer Flushing
When configured with
delimiter
, this plugin creates an internal buffer.When that buffer is present, this codec needs to implement
CEF#flush
so that the buffered bytes can be consumed when this codec is being flushed/closed.