[Filebeat] ASA Ingest Pipeline fixes (#26879) (#27921)

* #26869: ASA Ingest Pipeline fixes * Update changelog * Update 106023 & 302016 grok patterns * Update addtional grok patterns per comments * Fix per comment * update fields from master * Updated via last comment * Update pipeline * update pipeline to address #27137 (cherry picked from commit 5bc8572) Co-authored-by: Alex Resnick <adr8292@gmail.com>
elastic · Sep 14, 2021 · fe58408 · fe58408
1 parent fc6029f
commit fe58408
Show file tree

Hide file tree

Showing 24 changed files with 2,482 additions and 70 deletions.
diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc
@@ -434,6 +434,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Add `join` and `sprintf` functions to `httpjson` input. {pull}27735[27735]
 - Improve memory usage of line reader of `log` and `filestream` input. {pull}27782[27782]
 
+- Add `ignore_empty_value` flag to `httpjson` `split` processor. {pull}27880[27880]
+- Update Cisco ASA/FTD ingest pipeline grok/dissect patterns for multiple message IDs. {issue}26869[26869] {pull}26879[26879]
 
 *Heartbeat*
 

diff --git a/filebeat/docs/fields.asciidoc b/filebeat/docs/fields.asciidoc
@@ -21689,6 +21689,26 @@ type: keyword
 The WebVPN group name the user belongs to
 
 
+type: keyword
+
+--
+
+*`cisco.asa.termination_initiator`*::
++
+--
+Interface name of the side that initiated the teardown
+
+
+type: keyword
+
+--
+
+*`cisco.asa.tunnel_type`*::
++
+--
+SA type (remote access or L2L)
+
+
 type: keyword
 
 --
@@ -21927,6 +21947,16 @@ type: keyword
 The WebVPN group name the user belongs to
 
 
+type: keyword
+
+--
+
+*`cisco.ftd.termination_initiator`*::
++
+--
+Interface name of the side that initiated the teardown
+
+
 type: keyword
 
 --

diff --git a/x-pack/filebeat/module/cisco/asa/_meta/fields.yml b/x-pack/filebeat/module/cisco/asa/_meta/fields.yml
@@ -187,3 +187,15 @@
     default_field: false
     description: >
       The WebVPN group name the user belongs to
+
+  - name: termination_initiator
+    type: keyword
+    default_field: false
+    description: >
+      Interface name of the side that initiated the teardown
+
+  - name: tunnel_type
+    type: keyword
+    default_field: false
+    description: >
+      SA type (remote access or L2L)
diff --git a/x-pack/filebeat/module/cisco/asa/test/additional_messages.log b/x-pack/filebeat/module/cisco/asa/test/additional_messages.log
@@ -17,7 +17,7 @@ May  5 18:29:32 dev01: %ASA-6-302020: Built inbound ICMP connection for faddr 10
 May  5 18:29:32 dev01: %ASA-6-302020: Built outbound ICMP connection for faddr 10.10.10.10/0 gaddr 8.8.8.8/0 laddr 192.168.2.2/0 type 3 code 3
 May  5 18:29:32 dev01: %ASA-6-302014: Teardown TCP connection 2960892904 for out111:10.10.10.10/443 to fw111:192.168.2.2/55225 duration 0:00:00 bytes 0 TCP Reset-I
 May  5 18:29:32 dev01: %ASA-6-302013: Built outbound TCP connection 1588662 for intfacename:192.168.2.2/80 (8.8.8.8/80) to net:10.10.10.10/54839 (8.8.8.8/54839)
-May  5 18:29:32 dev01: %ASA-6-302012: Teardown dynamic UDP translation from fw111:10.10.10.10/54230 to out111:192.168.2.2/54230 duration 0:00:00
+May  5 18:29:32 dev01: %ASA-6-305012: Teardown dynamic UDP translation from fw111:10.10.10.10/54230 to out111:192.168.2.2/54230 duration 0:00:00
 May  5 18:40:50 dev01: %ASA-4-313004: Denied ICMP type=0, from laddr 10.10.10.10 on interface fw502 to 192.168.2.2: no matching session
 May  5 18:40:50 dev01: %ASA-6-305011: Built dynamic TCP translation from fw111:10.10.10.10/57006 to out111:192.168.2.2/57006
 May  5 18:40:50 dev01: %ASA-2-106001: Inbound TCP connection denied from 192.168.2.2/43803 to 10.10.10.10/14322 flags SYN  on interface out111
@@ -83,3 +83,10 @@ Apr 27 2020 02:03:03 dev01: %ASA-6-713904: All IPSec SA proposals found unaccept
 Apr 27 2020 02:03:03 dev01: %ASA-6-713903: IP = 192.128.1.1, All IPSec SA proposals found unacceptable!
 Apr 27 2020 02:03:03 dev01: %ASA-6-713902: Group = 100.60.140.10, All IPSec SA proposals found unacceptable!
 Apr 27 2020 02:03:03 dev01: %ASA-6-713901: Group = 100.60.140.10, IP = 192.128.1.1, All IPSec SA proposals found unacceptable!
+Apr 27 2020 02:03:03 dev01: %ASA-4-106023: Deny protocol 47 src outside:100.66.124.24 dst inside:172.31.98.44 by access-group "inbound"
+Apr 27 2020 02:03:03 dev01: %ASA-4-106023: Deny icmp src OUTSIDE:2a05:d016:add:4002:91f2:a9b2:e09a:6fc6 dst OUTSIDE:fe00:afa0::1 (type 128, code 0) by access-group "OUTSIDE_in"
+Apr 27 2020 02:03:03 dev01: %ASA-4-302016: Teardown UDP connection 123364823 for OUTSIDE:82.0.0.1/500 to identity:85.0.0.1/500 duration 92:24:20 bytes 4671944
+May  5 19:02:25 dev01: %ASA-4-733100: [ Scanning] drop rate-2 exceeded. Current burst rate is 0 per second, max configured rate is 8; Current average rate is 5 per second, max configured rate is 4; Cumulative total count is 19269
+May  5 19:02:25 dev01: %ASA-4-733100: [   192.168.0.1] drop rate-1 exceeded. Current burst rate is 0 per second, max configured rate is 10; Current average rate is 5 per second, max configured rate is 5; Cumulative total count is 6018
+May  5 19:02:25 dev01: %ASA-4-733100: [     Port-5432 5432] drop rate-1 exceeded. Current burst rate is 8 per second, max configured rate is 10; Current average rate is 20 per second, max configured rate is 5; Cumulative total count is 12466
+May  5 19:02:25 dev01: %ASA-4-733100: [           RDP 3389] drop rate-1 exceeded. Current burst rate is 63 per second, max configured rate is 10; Current average rate is 5 per second, max configured rate is 5; Cumulative total count is 3054