-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix Filebeat Zeek Weird Ingest Pipeline #15906
Conversation
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
1 similar comment
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
Hi @Xander33, thanks for contributing! Could you also add a test file with Zeek Weird logs which doesn't contain IP address please? Similar to |
Pinging @elastic/siem (Team:SIEM) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please also add an entry to the CHANGELOG.next.asciidoc file in the Filebeat bugfix section. Thanks
@andrewkroh good point. @kaiyan-sheng How does that look? |
This needs a quick rebase to resolve the conflicts. jenkins, test this |
jenkins, test this please |
Thanks @Xander33 for contributing! I will merge this PR and cherrypick to 7.x branch after CI passed. |
Some Zeek Weird logs do not contain IP addresses, causing the warning seen below:
Logstash Output
[2020-01-28T15:49:35,993][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-zeek-2020.01.28", :routing=>nil, :_type=>"_doc", :pipeline=>"filebeat-7.5.2-zeek-weird-pipeline"}, #<LogStash::Event:0x3f1f2270>], :response=>{"index"=>{"_index"=>"filebeat-zeek-2020.01.28", "_type"=>"_doc", "_id"=>"r3PX7G8BxFIJZtUR_Ruu", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [destination.ip] of type [ip] in document with id 'r3PX7G8BxFIJZtUR_Ruu'. Preview of field's value: ''", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'' is not an IP string literal."}}}}}
Sample from weird.log
{"ts":1580227259.342809,"name":"non_ip_packet_in_ethernet","notice":false,"peer":"ens3f1-4"}