-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Filebeat] Add Dashboards to Threat Intel Module #24488
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Trends 🧪💚 Flaky test reportTests succeeded. Expand to view the summary
Test stats 🧪
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
* added dashboards & docs * ran mage fmt update (cherry picked from commit 141273b)
* added dashboards & docs * ran mage fmt update (cherry picked from commit 141273b)
* upstream/master: [CI] bump gvm version and use the binary (elastic#24571) [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) Fix default scope in add_nomad_metadata (elastic#24559) [Heartbeat] Produce error rather than panic on missing source (elastic#24404) [Auditbeat] btmp offset check (elastic#24515) Clarify that the Tomcat module is for ingesting access logs (elastic#24543) [Ingest Manager] Move logging defaults to agent (elastic#24535)
…rchive-for-functionbeat * upstream/master: [CI] bump gvm version and use the binary (elastic#24571) [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) Fix default scope in add_nomad_metadata (elastic#24559)
…reporting-changes * upstream/master: [CI] bump gvm version and use the binary (elastic#24571) [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) Fix default scope in add_nomad_metadata (elastic#24559) [Heartbeat] Produce error rather than panic on missing source (elastic#24404)
* added dashboards & docs * ran mage fmt update
* mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (#24504) * Add tests for encoding settings of filestream input (#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (#24442) * Add syntax for multiple selector logging (#24207) (#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (#24446) * Add test for close.reader.after_interval to filestream input (#24423) * Refactor use of system.hostfs to fix cgroup metrics (#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (#24509) * Update input-http-endpoint.asciidoc (#24490) * [Ingest Manager] Move logging defaults to agent (#24535) [Ingest Manager] Move logging defaults to agent (#24535) * Clarify that the Tomcat module is for ingesting access logs (#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (#24404) Fixes #24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in #24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (#24571) * [CI] Add resilience when installing required tooling (#24542) * [CI] enable new flaky detector (#24464) * chore: do not pass beat version (#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com>
* mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (elastic#24504) * Add tests for encoding settings of filestream input (elastic#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (elastic#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (elastic#24442) * Add syntax for multiple selector logging (elastic#24207) (elastic#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (elastic#24446) * Add test for close.reader.after_interval to filestream input (elastic#24423) * Refactor use of system.hostfs to fix cgroup metrics (elastic#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (elastic#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (elastic#24509) * Update input-http-endpoint.asciidoc (elastic#24490) * [Ingest Manager] Move logging defaults to agent (elastic#24535) [Ingest Manager] Move logging defaults to agent (elastic#24535) * Clarify that the Tomcat module is for ingesting access logs (elastic#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (elastic#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (elastic#24404) Fixes elastic#24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in elastic#24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (elastic#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (elastic#24571) * [CI] Add resilience when installing required tooling (elastic#24542) * [CI] enable new flaky detector (elastic#24464) * chore: do not pass beat version (elastic#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit 5c6f1b6)
* mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (elastic#24504) * Add tests for encoding settings of filestream input (elastic#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (elastic#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (elastic#24442) * Add syntax for multiple selector logging (elastic#24207) (elastic#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (elastic#24446) * Add test for close.reader.after_interval to filestream input (elastic#24423) * Refactor use of system.hostfs to fix cgroup metrics (elastic#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (elastic#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (elastic#24509) * Update input-http-endpoint.asciidoc (elastic#24490) * [Ingest Manager] Move logging defaults to agent (elastic#24535) [Ingest Manager] Move logging defaults to agent (elastic#24535) * Clarify that the Tomcat module is for ingesting access logs (elastic#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (elastic#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (elastic#24404) Fixes elastic#24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in elastic#24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (elastic#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (elastic#24571) * [CI] Add resilience when installing required tooling (elastic#24542) * [CI] enable new flaky detector (elastic#24464) * chore: do not pass beat version (elastic#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit 5c6f1b6)
* mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (#24504) * Add tests for encoding settings of filestream input (#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (#24442) * Add syntax for multiple selector logging (#24207) (#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (#24446) * Add test for close.reader.after_interval to filestream input (#24423) * Refactor use of system.hostfs to fix cgroup metrics (#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (#24509) * Update input-http-endpoint.asciidoc (#24490) * [Ingest Manager] Move logging defaults to agent (#24535) [Ingest Manager] Move logging defaults to agent (#24535) * Clarify that the Tomcat module is for ingesting access logs (#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (#24404) Fixes #24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in #24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (#24571) * [CI] Add resilience when installing required tooling (#24542) * [CI] enable new flaky detector (#24464) * chore: do not pass beat version (#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit 5c6f1b6)
* Update gosigar package after fix (#24502) * mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (#24504) * Add tests for encoding settings of filestream input (#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (#24442) * Add syntax for multiple selector logging (#24207) (#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (#24446) * Add test for close.reader.after_interval to filestream input (#24423) * Refactor use of system.hostfs to fix cgroup metrics (#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (#24509) * Update input-http-endpoint.asciidoc (#24490) * [Ingest Manager] Move logging defaults to agent (#24535) [Ingest Manager] Move logging defaults to agent (#24535) * Clarify that the Tomcat module is for ingesting access logs (#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (#24404) Fixes #24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in #24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (#24571) * [CI] Add resilience when installing required tooling (#24542) * [CI] enable new flaky detector (#24464) * chore: do not pass beat version (#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit 5c6f1b6) * upadte changelog
…stic#24578) * added dashboards & docs * ran mage fmt update (cherry picked from commit 1e65129) Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com>
…lastic#24627) * Update gosigar package after fix (elastic#24502) * mofidy doc * work on fix * update changelog * update notice * Update notice * [Ingest Manager] Sync on rename on windows (elastic#24504) * Add tests for encoding settings of filestream input (elastic#24426) * [Elastic Agent] Add the ability to provide custom CA's inside of docker. (elastic#24486) * Add the ability to provide custom CA's for Elastic Agent docker. * Add changelog. * Update Golang to 1.15.9 (elastic#24442) * Add syntax for multiple selector logging (elastic#24207) (elastic#24497) * Add syntax for multiple selector logging * Update libbeat/docs/loggingconfig.asciidoc Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> * chore(ci): use beat_version instead of PR version (elastic#24446) * Add test for close.reader.after_interval to filestream input (elastic#24423) * Refactor use of system.hostfs to fix cgroup metrics (elastic#24334) * refactor use of system.hostfs to fix cgroup metrics * add changelog * remove comment * add cfgwarn * move changelog * shift around CLI config location and rep warning * add comment about system.hostfs usage * update docs * capitalization * fix grammar, add conditional * change docs phrasing * [Elastic Agent] Add verification check when updating communication to Kibana. (elastic#24489) * Add verification check when updating communication to Kibana. * Add changelog. * Add const. * Fix typo in mqtt input docs (elastic#24509) * Update input-http-endpoint.asciidoc (elastic#24490) * [Ingest Manager] Move logging defaults to agent (elastic#24535) [Ingest Manager] Move logging defaults to agent (elastic#24535) * Clarify that the Tomcat module is for ingesting access logs (elastic#24543) The Tomcat module is for ingesting access logs, not Catalina or localhost logs. * [Auditbeat] btmp offset check (elastic#24515) * auditbeat btmp offset check Add check that saved offset is not larger than the current file size to prevent seeking past the end of file * [Heartbeat] Produce error rather than panic on missing source (elastic#24404) Fixes elastic#24403. With the changes to the heartbeat config syntax in 7.12 the `source` field is now required. Our config validation code didn't actually check for this field's presence, which caused an NPE. This PR adds a validation checking for that config's presence. It also adds tests for the validation code for config sub-fields. There were no defects found in the validations for source.inline, or source.browser, but a few tests were missing. Instead of the panic seen in elastic#24403 users will now get the error seen below. ``` 2021-03-05T15:41:40.146-0600 ERROR instance/beat.go:952 Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') Exiting: could not create monitor: job err could not parse suite config: config 'source' must be specified for this monitor, if upgrading from a previous experimental version please see our new config docs accessing 'heartbeat.monitors.0' (source:'sample-synthetics-config/heartbeat.yml') ``` * Fix default scope in add_nomad_metadata (elastic#24559) Fix default scope in add_nomad_metadata. It is set to local, but it should be node. Fix also error message that showed that local is a valid value. * [Filebeat] Add Dashboards to Threat Intel Module (elastic#24488) * added dashboards & docs * ran mage fmt update * [CI] bump gvm version and use the binary (elastic#24571) * [CI] Add resilience when installing required tooling (elastic#24542) * [CI] enable new flaky detector (elastic#24464) * chore: do not pass beat version (elastic#24586) We will be delegating the version calculation to the e2e tests, using target branch values as defaults Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co> Co-authored-by: Michal Pristas <michal.pristas@gmail.com> Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com> Co-authored-by: Blake Rouse <blake.rouse@elastic.co> Co-authored-by: EamonnTP <Eamonn.Smith@elastic.co> Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co> Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> Co-authored-by: Alex K <8418476+fearful-symmetry@users.noreply.github.com> Co-authored-by: DeDe Morton <dede.morton@elastic.co> Co-authored-by: Lee Hinman <57081003+leehinman@users.noreply.github.com> Co-authored-by: Andrew Cholakian <andrew@andrewvc.com> Co-authored-by: Andrew Pease <7442091+peasead@users.noreply.github.com> Co-authored-by: Victor Martinez <victormartinezrubio@gmail.com> (cherry picked from commit b5b66f5) * upadte changelog
What does this PR do?
This PR adds a saved search, visualizations, and dashboards for the Threat Intel Filebeat Module.
Why is it important?
The overview dashboard provides valuable information on the health of the module, what type of data is being ingested, how much, and allows you to analyze module metrics.
The additional five dashboards provide a view into the different feeds and allow an analyst to hunt and find connections and associations that may not have previously been exposed.
Checklist
CHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.Author's Checklist
./filebeat setup
imports the saved search, visualization, and dashboardHow to test this PR locally
This can be tested by:
Go to KIbana -> dashboards -> apply the
threat intel
tagRelated issues
Resolves #24487
Use cases
Threat hunting, security operations, and intelligence analysis.
Screenshots
Threat Intel Overview
Threat Intel Abuse Malware
Threat Intel Abuse URL
Threat Intel AlienVault OTX
Threat Intel Anomali Limo
Threat Intel MISP
Logs