diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 55a0594e80b..170db0d809b 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -3,6 +3,127 @@ :issue: https://github.com/elastic/beats/issues/ :pull: https://github.com/elastic/beats/pull/ +[[release-notes-7.13.0]] +=== Beats version 7.13.0 +https://github.com/elastic/beats/compare/v7.12.1...v7.13.0[View commits] + +==== Breaking changes + +*Affecting all Beats* + +- Use alias to report container image in k8s metadata. {pull}24380[24380] +- Set `cleanup_timeout` to zero by default in docker and kubernetes autodiscover in all beats except Filebeat where it is kept to 60 seconds. {pull}24681[24681] +- Update to ECS 1.9.0. {pull}24909[24909] + +*Filebeat* + +- Changes filebeat httpjson input's append transform to create a list even with only a single value{pull}25074[25074] +- Deprecated the cyberark module (replaced by cyberarkpas). {issue}25261[25261] {pull}25505[25505] + +*Metricbeat* + +- Store `cloudfoundry.container.cpu.pct` in decimal form and as `scaled_float`. {pull}24219[24219] +- Remove `index_stats.created` field from Elasticsearch/index Metricset {pull}25113[25113] + +==== Bugfixes + +*Affecting all Beats* + +- Fix events being dropped if they contain a floating point value of NaN or Inf. {pull}25051[25051] +- Fix templates being overwritten if there was an error when check for the template existance. {pull}24332[24332] +- Add `expand_keys` to the list of permitted config fields for `decode_json_fields` {24862}[24862] +- Fix discovery of short-living and failing pods in Kubernetes autodiscover {issue}22718[22718] {pull}24742[24742] +- Fix panic when overwriting metadata {pull}24741[24741] +- Fix role_arn to work with access keys for AWS. {pull}25446[25446] +- Fix `community_id` processor so that ports greater than 65535 aren't valid. {pull}25409[25409] + +*Auditbeat* + +- Fix o365 module config when client_secret contains special characters. {issue}25058[25058] + +*Filebeat* + +- Fix date parsing in GSuite/login fileset. {issue}24694[24694] +- Improve Cisco ASA/FTD parsing of messages {pull}23766[23766] + - Better support for identity FW messages. + - Change network.bytes, source.bytes, and destination.bytes to long from integer since value can exceed integer capacity. + - Add descriptions for various processors for easier pipeline editing in Kibana UI. +- Fix usage of unallowed ECS event.outcome values in Cisco ASA/FTD pipeline. {pull}24744[24744]. +- Fix IPtables Pipeline and Ubiquiti dashboard. {issue}24878[24878] {pull}24928[24928] +- Strip Azure Eventhub connection string in debug logs. {pulll}25066[25066] +- Updating Oauth2 flow for m365_defender fileset. {pull}24829[24829] +- Fix o365 module config when client_secret contains special characters. {issue}25058[25058] +- Fix s3 input when there is a blank line in the log file. {pull}25357[25357] +- Remove space from field `sophos.xg.trans_src_ ip`. {issue}25154[25154] {pull}25250[25250] +- Fix `checkpoint.action_reason` when its a string, not a Long. {issue}25575[25575] {pull}25609[25609] +- Fix `fortinet.firewall.addr` when its a string, not an IP address. {issue}25585[25585] {pull}25608[25608] + +*Metricbeat* + +- Sort correctly the keys when accessing JMX through the Jolokia module {pull}25631[25631] +- Change lookup_fields from metricset.host to service.address {pull}15883[15883] +- Fix incorrect types of fields GetHits and Ops in NodeInterestingStats for Couchbase module in Metricbeat {issue}21021[21021] {pull}23287[23287] +- Fix GCP not able to request Cloudfunctions metrics if a region filter was set {pull}24218[24218] +- Fix type of `uwsgi.status.worker.rss` type. {pull}24468[24468] +- Accept text/plain type by default for prometheus client scraping. {pull}24622[24622] +- Use working set bytes to calculate the pod memory limit pct when memory usage is not reported (ie. Windows pods). {pull}25428[25428] +- Fix copy-paste error in libbeat docs. {pull}25448[25448] +- Fix azure billing dashboard. {pull}25554[25554] + +*Winlogbeat* + +- Change `event.code` and `winlog.event_id` from int to keyword. {pull}25176[25176] + +==== Added + +*Affecting all Beats* + +- Add `wineventlog` schema to `decode_xml` processor. {issue}23910[23910] {pull}24726[24726] +- Add new ECS 1.9 field `cloud.service.name` to `add_cloud_metadata` processor. {pull}24993[24993] +- Libbeat: report queue capacity, output batch size, and output client count to monitoring. {pull}24700[24700] +- Add kubernetes.pod.ip field in kubernetes metadata. {pull}25037[25037] +- Discover changes in Kubernetes namespace metadata as soon as they happen. {pull}25117[25117] +- Add `decode_xml_wineventlog` processor. {issue}23910[23910] {pull}25115[25115] +- Add new setting `gc_percent` for tuning the garbage collector limits via configuration file. {pull}25394[25394] +- Add `unit` and `metric_type` properties to fields.yml for populating field metadata in Elasticsearch templates {pull}25419[25419] +- Add new option `suffix` to `logging.files` to control how log files are rotated. {pull}25464[25464] +- Validate that required functionality in Elasticsearch is available upon initial connection. {pull}25351[25351] + +*Filebeat* + +- Support X-Forwarder-For in IIS logs. {pull}19142[192142] +- Add support for logs generated by servers configured with `log_statement` and `log_duration` in PostgreSQL module. {pull}24607[24607] +- Added fifteen new message IDs to Cisco ASA/FTD pipeline. {pull}24744[24744] +- Added NTP fileset to Zeek module {pull}24224[24224] +- Add `proxy_url` config for httpjson v2 input. {issue}24615[24615] {pull}24662[24662] +- Change `okta.target` to `flattened` field type. {issue}24354[24354] {pull}24636[24636] +- Added `http.request.id` to `nginx/ingress_controller` and `elasticsearch/audit`. {pull}24994[24994] +- Add `awsfargate` module to collect container logs from Amazon ECS on Fargate. {pull}25041[25041] +- New module `cyberarkpas` for CyberArk Privileged Access Security audit logs. {pull}24803[24803] +- Add `uri_parts` processor to Apache, Nginx, IIS, Traefik, S3Access, Cisco, F5, Fortinet, Google Workspace, Imperva, Microsoft, Netscout, O365, Sophos, Squid, Suricata, Zeek, Zia, Zoom, and ZScaler modules ingest pipelines. {issue}19088[19088] {pull}24699[24699] +- New module `zookeeper` for Zookeeper service and audit logs {issue}25061[25061] {pull}25128[25128] +- Add parsing for `haproxy.http.request.raw_request_line` field {issue}25480[25480] {pull}25482[25482] +- Mark `filestream` input beta. {pull}25560[25560] +- Add User Agent Parser for Azure Sign In Logs Ingest Pipeline {pull}23201[23201] + +*Heartbeat* + +- Handle datastreams for fleet. {pull}24223[24223] +- Add --sandbox option for browser monitor. {pull}24172[24172] +- Support additional 'root' fields from synthetics. {pull}24770[24770] +- Browser zip_url source type. {pull}24714[24714] + +*Metricbeat* + +- Add support for Consul 1.9. {pull}24123[24123] +- Add support for defining metrics_filters for prometheus module in hints. {pull}24264[24264] +- Add support for PostgreSQL 10, 11, 12 and 13. {pull}24402[24402] +- Add support for SASL/SCRAM authentication to the Kafka module. {pull}24810[24810] + +*Winlogbeat* + +- Add support for sysmon v13 events 24 and 25. {issue}24217[24217] {pull}24945[24945] + [[release-notes-7.12.1]] === Beats version 7.12.1 https://github.com/elastic/beats/compare/v7.12.0...v7.12.1[View commits] diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index 74bf57b0156..74f186ece9e 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -35,6 +35,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Set `cleanup_timeout` to zero by default in docker and kubernetes autodiscover in all beats except Filebeat where it is kept to 60 seconds. {pull}24681[24681] - Update to ECS 1.9.0. {pull}24909[24909] - Remove id_field_data {pull}25239[25239] +- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820] *Auditbeat* @@ -414,18 +415,10 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d *Metricbeat* -- Sort correctly the keys when accessing JMX through the Jolokia module {pull}25631[25631] -- Add dedot for tags in ec2 metricset and cloudwatch metricset. {issue}15843[15843] {pull}15844[15844] -- Use RFC3339 format for timestamps collected using the SQL module. {pull}15847[15847] -- Avoid parsing errors returned from prometheus endpoints. {pull}15712[15712] -- Change lookup_fields from metricset.host to service.address {pull}15883[15883] -- Add dedot for cloudwatch metric name. {issue}15916[15916] {pull}15917[15917] -- Fixed issue `logstash-xpack` module suddenly ceasing to monitor Logstash. {issue}15974[15974] {pull}16044[16044] - Fix checking tagsFilter using length in cloudwatch metricset. {pull}14525[14525] - Fixed bug with `elasticsearch/cluster_stats` metricset not recording license expiration date correctly. {issue}14541[14541] {pull}14591[14591] - Log bulk failures from bulk API requests to monitoring cluster. {issue}14303[14303] {pull}14356[14356] - Fixed bug with `elasticsearch/cluster_stats` metricset not recording license ID in the correct field. {pull}14592[14592] -- Change lookup_fields from metricset.host to service.address {pull}15883[15883] - Fix skipping protocol scheme by light modules. {pull}16205[pull] - Made `logstash-xpack` module once again have parity with internally-collected Logstash monitoring data. {pull}16198[16198] - Revert changes in `docker` module: add size flag to docker.container. {pull}16600[16600] @@ -842,6 +835,32 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add Google Workspace module and mark Gsuite module as deprecated {pull}22950[22950] - Mark m365 defender, defender atp, okta and google workspace modules as GA {pull}23113[23113] - Added `alternative_host` option to google pubsub input {pull}23215[23215] +- Added support for first_event context in filebeat httpjson input {pull}23437[23437] +- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by + removing unsupported processors. {pull}23763[23763] +- Added support for Cisco AMP API as a new fileset. {pull}22768[22768] +- Added RFC6587 framing option for tcp and unix inputs {issue}23663[23663] {pull}23724[23724] +- Added `application/x-ndjson` as decode option for httpjson input {pull}23521[23521] +- Added `application/x-www-form-urlencoded` as encode option for httpjson input {pull}23521[23521] +- Move aws-s3 input to GA. {pull}23631[23631] +- Populate `source.mac` and `destination.mac` for Suricata EVE events. {issue}23706[23706] {pull}23721[23721] +- Added string splitting for httpjson input {pull}24022[24022] +- Added Signatures fileset to Zeek module {pull}23772[23772] +- Upgrade Cisco ASA/FTD/Umbrella to ECS 1.8.0. {pull}23819[23819] +- Add new ECS user and categories features to google_workspace/gsuite {issue}23118[23118] {pull}23709[23709] +- Move crowdstrike JS processor to ingest pipelines and upgrade to ECS 1.8.0 {issue}23118[23118] {pull}23875[23875] +- Update Filebeat auditd dataset to ECS 1.8.0. {pull}23723[23723] {issue}23118[23118] +- Updated microsoft defender_atp and m365_defender to ECS 1.8. {pull}23897[23897] {issue}23118[23118] +- Updated o365 module to ECS 1.8. {issue}23118[23118] {pull}23896[23896] +- Upgrade CEF module to ECS 1.8.0. {pull}23832[23832] +- Upgrade fortinet/firewall to ECS 1.8 {issue}23118[23118] {pull}23902[23902] +- Upgrade Zeek to ECS 1.8.0. {issue}23118[23118] {pull}23847[23847] +- Updated azure module to ECS 1.8. {issue}23118[23118] {pull}23927[23927] +- Update aws/s3access to ECS 1.8. {issue}23118[23118] {pull}23920[23920] +- Upgrade panw module to ecs 1.8 {issue}23118[23118] {pull}23931[23931] +- Upgrade juniper/srx to ecs 1.8.0. {issue}23118[23118] {pull}23936[23936] +- Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] +- Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118] - Support X-Forwarder-For in IIS logs. {pull}19142[192142] - Add support for logs generated by servers configured with `log_statement` and `log_duration` in PostgreSQL module. {pull}24607[24607] - Added fifteen new message IDs to Cisco ASA/FTD pipeline. {pull}24744[24744] @@ -869,6 +888,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add --sandbox option for browser monitor. {pull}24172[24172] - Support additional 'root' fields from synthetics. {pull}24770[24770] - Browser zip_url source type. {pull}24714[24714] +- Bundle synthetics deps with heartbeat docker image. {pull}23274[23274] *Journalbeat* @@ -992,6 +1012,10 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Move IIS module to GA and map fields. {issue}22609[22609] {pull}23024[23024] - Apache: convert status.total_kbytes to status.total_bytes in fleet mode. {pull}23022[23022] - Release MSSQL as GA {pull}23146[23146] +- Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] +- Check fields are documented in aws metricsets. {pull}23887[23887] +- Add support for defining metrics_filters for prometheus module in hints. {pull}24264[24264] +- Add support for PostgreSQL 10, 11, 12 and 13. {pull}24402[24402] - Add support for SASL/SCRAM authentication to the Kafka module. {pull}24810[24810] - Refactor state_* metricsets to share response from endpoint. {pull}25640[25640] - Add server id to zookeeper events. {pull}25550[25550] diff --git a/libbeat/docs/release.asciidoc b/libbeat/docs/release.asciidoc index 16c5f75d2bf..43387e8c877 100644 --- a/libbeat/docs/release.asciidoc +++ b/libbeat/docs/release.asciidoc @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read <> for more detail about changes that affect upgrade. +* <> * <> * <> * <>