Skip to content

Commit

Permalink
Forward port to 7.x: Prepare Changelog for 7.13.0 (elastic#25823) (el…
Browse files Browse the repository at this point in the history 
…astic#25849)

* docs: Prepare Changelog for 7.13.0 (elastic#25823)

* docs: Close changelog for 7.13.0

* Cleanup

* Cleanup

* Move 23201 to the correct place

* Apply suggestions from code review

Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>

* Move 24123

* Remove empty sections

* Add missing 7.12.1

* Additional fixes

Co-authored-by: Andres Rodriguez <andresrc@gmail.com>
Co-authored-by: Andres Rodriguez <andres.rodriguez@elastic.co>
Co-authored-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
(cherry picked from commit b332fc2)

* Fix index

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
  • Loading branch information
@andresrc @elasticmachine
andresrc and elasticmachine authored May 25, 2021
1 parent 6814937 commit f5e853f
Show file tree
Hide file tree
Showing 3 changed files with 122 additions and 78 deletions.
121 changes: 121 additions & 0 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,127 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-7.13.0]]
=== Beats version 7.13.0
https://github.com/elastic/beats/compare/v7.12.1...v7.13.0[View commits]

==== Breaking changes

*Affecting all Beats*

- Use alias to report container image in k8s metadata. {pull}24380[24380]
- Set `cleanup_timeout` to zero by default in docker and kubernetes autodiscover in all beats except Filebeat where it is kept to 60 seconds. {pull}24681[24681]
- Update to ECS 1.9.0. {pull}24909[24909]

*Filebeat*

- Changes filebeat httpjson input's append transform to create a list even with only a single value{pull}25074[25074]
- Deprecated the cyberark module (replaced by cyberarkpas). {issue}25261[25261] {pull}25505[25505]

*Metricbeat*

- Store `cloudfoundry.container.cpu.pct` in decimal form and as `scaled_float`. {pull}24219[24219]
- Remove `index_stats.created` field from Elasticsearch/index Metricset {pull}25113[25113]

==== Bugfixes

*Affecting all Beats*

- Fix events being dropped if they contain a floating point value of NaN or Inf. {pull}25051[25051]
- Fix templates being overwritten if there was an error when check for the template existance. {pull}24332[24332]
- Add `expand_keys` to the list of permitted config fields for `decode_json_fields` {24862}[24862]
- Fix discovery of short-living and failing pods in Kubernetes autodiscover {issue}22718[22718] {pull}24742[24742]
- Fix panic when overwriting metadata {pull}24741[24741]
- Fix role_arn to work with access keys for AWS. {pull}25446[25446]
- Fix `community_id` processor so that ports greater than 65535 aren't valid. {pull}25409[25409]

*Auditbeat*

- Fix o365 module config when client_secret contains special characters. {issue}25058[25058]

*Filebeat*

- Fix date parsing in GSuite/login fileset. {issue}24694[24694]
- Improve Cisco ASA/FTD parsing of messages {pull}23766[23766]
- Better support for identity FW messages.
- Change network.bytes, source.bytes, and destination.bytes to long from integer since value can exceed integer capacity.
- Add descriptions for various processors for easier pipeline editing in Kibana UI.
- Fix usage of unallowed ECS event.outcome values in Cisco ASA/FTD pipeline. {pull}24744[24744].
- Fix IPtables Pipeline and Ubiquiti dashboard. {issue}24878[24878] {pull}24928[24928]
- Strip Azure Eventhub connection string in debug logs. {pulll}25066[25066]
- Updating Oauth2 flow for m365_defender fileset. {pull}24829[24829]
- Fix o365 module config when client_secret contains special characters. {issue}25058[25058]
- Fix s3 input when there is a blank line in the log file. {pull}25357[25357]
- Remove space from field `sophos.xg.trans_src_ ip`. {issue}25154[25154] {pull}25250[25250]
- Fix `checkpoint.action_reason` when its a string, not a Long. {issue}25575[25575] {pull}25609[25609]
- Fix `fortinet.firewall.addr` when its a string, not an IP address. {issue}25585[25585] {pull}25608[25608]

*Metricbeat*

- Sort correctly the keys when accessing JMX through the Jolokia module {pull}25631[25631]
- Change lookup_fields from metricset.host to service.address {pull}15883[15883]
- Fix incorrect types of fields GetHits and Ops in NodeInterestingStats for Couchbase module in Metricbeat {issue}21021[21021] {pull}23287[23287]
- Fix GCP not able to request Cloudfunctions metrics if a region filter was set {pull}24218[24218]
- Fix type of `uwsgi.status.worker.rss` type. {pull}24468[24468]
- Accept text/plain type by default for prometheus client scraping. {pull}24622[24622]
- Use working set bytes to calculate the pod memory limit pct when memory usage is not reported (ie. Windows pods). {pull}25428[25428]
- Fix copy-paste error in libbeat docs. {pull}25448[25448]
- Fix azure billing dashboard. {pull}25554[25554]

*Winlogbeat*

- Change `event.code` and `winlog.event_id` from int to keyword. {pull}25176[25176]

==== Added

*Affecting all Beats*

- Add `wineventlog` schema to `decode_xml` processor. {issue}23910[23910] {pull}24726[24726]
- Add new ECS 1.9 field `cloud.service.name` to `add_cloud_metadata` processor. {pull}24993[24993]
- Libbeat: report queue capacity, output batch size, and output client count to monitoring. {pull}24700[24700]
- Add kubernetes.pod.ip field in kubernetes metadata. {pull}25037[25037]
- Discover changes in Kubernetes namespace metadata as soon as they happen. {pull}25117[25117]
- Add `decode_xml_wineventlog` processor. {issue}23910[23910] {pull}25115[25115]
- Add new setting `gc_percent` for tuning the garbage collector limits via configuration file. {pull}25394[25394]
- Add `unit` and `metric_type` properties to fields.yml for populating field metadata in Elasticsearch templates {pull}25419[25419]
- Add new option `suffix` to `logging.files` to control how log files are rotated. {pull}25464[25464]
- Validate that required functionality in Elasticsearch is available upon initial connection. {pull}25351[25351]

*Filebeat*

- Support X-Forwarder-For in IIS logs. {pull}19142[192142]
- Add support for logs generated by servers configured with `log_statement` and `log_duration` in PostgreSQL module. {pull}24607[24607]
- Added fifteen new message IDs to Cisco ASA/FTD pipeline. {pull}24744[24744]
- Added NTP fileset to Zeek module {pull}24224[24224]
- Add `proxy_url` config for httpjson v2 input. {issue}24615[24615] {pull}24662[24662]
- Change `okta.target` to `flattened` field type. {issue}24354[24354] {pull}24636[24636]
- Added `http.request.id` to `nginx/ingress_controller` and `elasticsearch/audit`. {pull}24994[24994]
- Add `awsfargate` module to collect container logs from Amazon ECS on Fargate. {pull}25041[25041]
- New module `cyberarkpas` for CyberArk Privileged Access Security audit logs. {pull}24803[24803]
- Add `uri_parts` processor to Apache, Nginx, IIS, Traefik, S3Access, Cisco, F5, Fortinet, Google Workspace, Imperva, Microsoft, Netscout, O365, Sophos, Squid, Suricata, Zeek, Zia, Zoom, and ZScaler modules ingest pipelines. {issue}19088[19088] {pull}24699[24699]
- New module `zookeeper` for Zookeeper service and audit logs {issue}25061[25061] {pull}25128[25128]
- Add parsing for `haproxy.http.request.raw_request_line` field {issue}25480[25480] {pull}25482[25482]
- Mark `filestream` input beta. {pull}25560[25560]
- Add User Agent Parser for Azure Sign In Logs Ingest Pipeline {pull}23201[23201]

*Heartbeat*

- Handle datastreams for fleet. {pull}24223[24223]
- Add --sandbox option for browser monitor. {pull}24172[24172]
- Support additional 'root' fields from synthetics. {pull}24770[24770]
- Browser zip_url source type. {pull}24714[24714]

*Metricbeat*

- Add support for Consul 1.9. {pull}24123[24123]
- Add support for defining metrics_filters for prometheus module in hints. {pull}24264[24264]
- Add support for PostgreSQL 10, 11, 12 and 13. {pull}24402[24402]
- Add support for SASL/SCRAM authentication to the Kafka module. {pull}24810[24810]

*Winlogbeat*

- Add support for sysmon v13 events 24 and 25. {issue}24217[24217] {pull}24945[24945]

[[release-notes-7.12.1]]
=== Beats version 7.12.1
https://github.com/elastic/beats/compare/v7.12.0...v7.12.1[View commits]
Expand Down
78 changes: 0 additions & 78 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Introduce APM libbeat instrumentation, active when running the beat with ELASTIC_APM_ACTIVE=true. {pull}17938[17938]
- Make error message about locked data path actionable. {pull}18667[18667]
- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820]
- Use alias to report container image in k8s metadata. {pull}24380[24380]
- Set `cleanup_timeout` to zero by default in docker and kubernetes autodiscover in all beats except Filebeat where it is kept to 60 seconds. {pull}24681[24681]
- Update to ECS 1.9.0. {pull}24909[24909]

*Auditbeat*

Expand Down Expand Up @@ -291,18 +288,10 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d

*Metricbeat*

- Sort correctly the keys when accessing JMX through the Jolokia module {pull}25631[25631]
- Add dedot for tags in ec2 metricset and cloudwatch metricset. {issue}15843[15843] {pull}15844[15844]
- Use RFC3339 format for timestamps collected using the SQL module. {pull}15847[15847]
- Avoid parsing errors returned from prometheus endpoints. {pull}15712[15712]
- Change lookup_fields from metricset.host to service.address {pull}15883[15883]
- Add dedot for cloudwatch metric name. {issue}15916[15916] {pull}15917[15917]
- Fixed issue `logstash-xpack` module suddenly ceasing to monitor Logstash. {issue}15974[15974] {pull}16044[16044]
- Fix checking tagsFilter using length in cloudwatch metricset. {pull}14525[14525]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license expiration date correctly. {issue}14541[14541] {pull}14591[14591]
- Log bulk failures from bulk API requests to monitoring cluster. {issue}14303[14303] {pull}14356[14356]
- Fixed bug with `elasticsearch/cluster_stats` metricset not recording license ID in the correct field. {pull}14592[14592]
- Change lookup_fields from metricset.host to service.address {pull}15883[15883]
- Fix skipping protocol scheme by light modules. {pull}16205[pull]
- Made `logstash-xpack` module once again have parity with internally-collected Logstash monitoring data. {pull}16198[16198]
- Revert changes in `docker` module: add size flag to docker.container. {pull}16600[16600]
Expand Down Expand Up @@ -507,46 +496,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Keep cursor state between httpjson input restarts {pull}20751[20751]
- New juniper.srx dataset for Juniper SRX logs. {pull}20017[20017]
- Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291]
- Added TLS JA3 fingerprint, certificate not_before/not_after, certificate SHA1 hash, and certificate subject fields to Zeek SSL dataset. {pull}21696[21696]
- Add platform logs in the azure filebeat module. {pull}22371[22371]
- Added `event.ingested` field to data from the Netflow module. {pull}22412[22412]
- Improve panw ECS url fields mapping. {pull}22481[22481]
- Improve Nats filebeat dashboard. {pull}22726[22726]
- Add support for UNIX datagram sockets in `unix` input. {issues}18632[18632] {pull}22699[22699]
- Add `http.request.mime_type` for Elasticsearch audit log fileset. {pull}22975[22975]
- Add new httpjson input features and mark old config ones for deprecation {pull}22320[22320]
- Add configuration option to set external and internal networks for panw panos fileset {pull}22998[22998]
- Add `subbdomain` fields for rsa2elk modules. {pull}23035[23035]
- Add subdomain enrichment for suricata/eve fileset. {pull}23011[23011]
- Add subdomain enrichment for zeek/dns fileset. {pull}23011[23011]
- Add `event.category` "configuration" to auditd module events. {pull}23010[23010]
- Add `event.category` "configuration" to gsuite module events. {pull}23010[23010]
- Add `event.category` "configuration" to o365 module events. {pull}23010[23010]
- Add `event.category` "configuration" to zoom module events. {pull}23010[23010]
- Add `network.direction` to auditd/log fileset. {pull}23041[23041]
- Add logic for external network.direction in sophos xg fileset {pull}22973[22973]
- Preserve AWS CloudTrail eventCategory in aws.cloudtrail.event_category. {issue}22776[22776] {pull}22805[22805]
- Add top_level_domain enrichment for suricata/eve fileset. {pull}23046[23046]
- Add top_level_domain enrichment for zeek/dns fileset. {pull}23046[23046]
- Add `observer.egress.zone` and `observer.ingress.zone` for cisco/asa and cisco/ftd filesets. {pull}23068[23068]
- Allow cisco/asa and cisco/ftd filesets to override network directionality based off of zones. {pull}23068[23068]
- Allow cef and checkpoint modules to override network directionality based off of zones {pull}23066[23066]
- Add `network.direction` to netflow/log fileset. {pull}23052[23052]
- Add the ability to override `network.direction` based on interfaces in Fortinet/firewall fileset. {pull}23072[23072]
- Add `network.direction` override by specifying `internal_networks` in gcp module. {pull}23081[23081]
- Migrate microsoft/defender_atp to httpjson v2 config {pull}23017[23017]
- Migrate microsoft/m365_defender to httpjson v2 config {pull}23018[23018]
- Migrate okta to httpjson v2 config {pull}23059[23059]
- Add support for Snyk Vulnerability and Audit API. {pull}22677[22677]
- Misp improvements: Migration to httpjson v2 config, pagination and deduplication ID {pull}23070[23070]
- Add Google Workspace module and mark Gsuite module as deprecated {pull}22950[22950]
- Mark m365 defender, defender atp, okta and google workspace modules as GA {pull}23113[23113]
- Add parsing of tcp flags to AWS vpcflow fileset {issue}228020[22820] {pull}23157[23157]
- Added support for first_event context in filebeat httpjson input {pull}23437[23437]
- Added `alternative_host` option to google pubsub input {pull}23215[23215]
- Adding Threat Intel module {pull}21795[21795]
- Added username parsing from Cisco ASA message 302013. {pull}21196[21196]
- Added `encode_as` and `decode_as` options to httpjson along with pluggable encoders/decoders {pull}23478[23478]
- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by
removing unsupported processors. {pull}23763[23763]
- Added support for Cisco AMP API as a new fileset. {pull}22768[22768]
Expand All @@ -569,12 +519,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Updated azure module to ECS 1.8. {issue}23118[23118] {pull}23927[23927]
- Update aws/s3access to ECS 1.8. {issue}23118[23118] {pull}23920[23920]
- Upgrade panw module to ecs 1.8 {issue}23118[23118] {pull}23931[23931]
- Updated aws/cloudtrail fileset to ECS 1.8. {issue}23118[23118] {pull}23911[23911]
- Upgrade juniper/srx to ecs 1.8.0. {issue}23118[23118] {pull}23936[23936]
- Update mysqlenterprise module to ECS 1.8. {issue}23118[23118] {pull}23978[23978]
- Upgrade sophos/xg fileset to ECS 1.8.0. {issue}23118[23118] {pull}23967[23967]
- Upgrade system/auth to ECS 1.8 {issue}23118[23118] {pull}23961[23961]
- Upgrade elasticsearch/audit to ECS 1.8 {issue}23118[23118] {pull}24000[24000]
- Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929]
- Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118]
- Support X-Forwarder-For in IIS logs. {pull}19142[192142]
Expand All @@ -598,12 +543,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d

*Heartbeat*

- Add mime type detection for http responses. {pull}22976[22976]
- Bundle synthetics deps with heartbeat docker image. {pull}23274[23274]
- Handle datastreams for fleet. {pull}24223[24223]
- Add --sandbox option for browser monitor. {pull}24172[24172]
- Support additional 'root' fields from synthetics. {pull}24770[24770]
- Browser zip_url source type. {pull}24714[24714]

*Heartbeat*

Expand Down Expand Up @@ -665,25 +605,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add billing metricset into googlecloud module. {pull}20812[20812] {issue}20738[20738]
- Release lambda metricset in aws module as GA. {issue}21251[21251] {pull}21255[21255]
- Add dashboard for pubsub metricset in googlecloud module. {pull}21326[21326] {issue}17137[17137]
- Move Prometheus query & remote_write to GA. {pull}21507[21507]
- Map cloud data filed `cloud.account.id` to azure subscription. {pull}21483[21483] {issue}21381[21381]
- Expand unsupported option from namespace to metrics in the azure module. {pull}21486[21486]
- Move s3_daily_storage and s3_request metricsets to use cloudwatch input. {pull}21703[21703]
- Duplicate system.process.cmdline field with process.command_line ECS field name. {pull}22325[22325]
- Add awsfargate module task_stats metricset to monitor AWS ECS Fargate. {pull}22034[22034]
- Add connection and route metricsets for nats metricbeat module to collect metrics per connection/route. {pull}22445[22445]
- Add unit file states to system/service {pull}22557[22557]
- `kibana` module: `stats` metricset no-longer collects usage-related data. {pull}22732[22732]
- Add more TCP states to Metricbeat system socket_summary. {pull}14347[14347]
- Add io.ops in fields exported by system.diskio. {pull}22066[22066]
- Adjust the Apache status fields in the fleet mode. {pull}22821[22821]
- Add AWS Fargate overview dashboard. {pull}22941[22941]
- Add process.state, process.cpu.pct, process.cpu.start_time and process.memory.pct. {pull}22845[22845]
- Move IIS module to GA and map fields. {issue}22609[22609] {pull}23024[23024]
- Apache: convert status.total_kbytes to status.total_bytes in fleet mode. {pull}23022[23022]
- Release MSSQL as GA {pull}23146[23146]
- Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730]
- Add support for Darwin/arm M1. {pull}24019[24019]
- Check fields are documented in aws metricsets. {pull}23887[23887]
- Add support for defining metrics_filters for prometheus module in hints. {pull}24264[24264]
- Add support for PostgreSQL 10, 11, 12 and 13. {pull}24402[24402]
Expand Down
1 change: 1 addition & 0 deletions libbeat/docs/release.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read
<<breaking-changes>> for more detail about changes that affect
upgrade.

* <<release-notes-7.13.0>>
* <<release-notes-7.12.1>>
* <<release-notes-7.12.0>>
* <<release-notes-7.11.2>>
Expand Down

0 comments on commit f5e853f

Please sign in to comment.