From 3d3f96e71634708151a16a750c061c317f6957a2 Mon Sep 17 00:00:00 2001 From: elasticmachine Date: Fri, 19 Mar 2021 16:28:37 +0000 Subject: [PATCH 01/10] docs: Close changelog for 7.12.0 --- CHANGELOG.asciidoc | 232 ++++++++++++++++++++++++++++++ CHANGELOG.next.asciidoc | 262 ++-------------------------------- libbeat/docs/release.asciidoc | 1 + 3 files changed, 242 insertions(+), 253 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 51c0f1f8aea3..e3a3bdee26f2 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -3,6 +3,238 @@ :issue: https://github.com/elastic/beats/issues/ :pull: https://github.com/elastic/beats/pull/ +[[release-notes-7.12.0]] +=== Beats version 7.12.0 +https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] + +==== Breaking changes + +*Affecting all Beats* + +- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820] + +*Auditbeat* + +*Filebeat* + +- Add fileset to ingest PostgreSQL CSV logs. {pull}23334[23334] +- Rename `s3` input to `aws-s3` input. {pull}23469[23469] + +*Heartbeat* +- Refactor synthetics configuration to new syntax. {pull}23467[23467] + +*Journalbeat* + + + +*Metricbeat* + +- Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] +- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}[23905] + +*Packetbeat* + + +*Winlogbeat* + + +*Functionbeat* + + +==== Bugfixes + +*Affecting all Beats* + +- Fix `nested` subfield handling in generated Elasticsearch templates. {issue}23178[23178] {pull}23183[23183] +- Fix CPU usage metrics on VMs with dynamic CPU config {pull}23154[23154] +- Allow configuring credential_profile_name and shared_credential_file when using role_arn. {pull}24174[24174] + + +*Auditbeat* + +- system/login: Fixed offset reset on inode reuse. {pull}24414[24414] +- system/login: Add additional offset check for utmp files. {pull}24515[24515] + +*Filebeat* + +- CheckPoint Firewall module: Change event.severity JSON data type to a number because the field mapping is a `long`. {pull}23424[23424] +- Cisco IOS: Change icmp.type/code and igmp.type JSON data types to strings because the fields mappings are `keyword`. {pull}23424[23424] +- CrowdStrike Falcon: Change JSON field types to match the field mappings. {pull}23424[23424] +- Fortinet Firewall: Drop `fortinet.firewall.assignip` when the value is "N/A". {pull}23424[23424] +- Juniper SRX: Change JSON field types to match the field mappings. {pull}23424[23424] +- Suricata EVE: Convert `suricata.eve.flow_id` to string because the field is a keyword in the mapping. {pull}23424[23424] +- Zeek DNS: Ignore failures in data type conversions. And change `dns.id` JSON field to a string to match its `keyword` mapping. {pull}23424[23424] +- Update `filestream` reader offset when a line is skipped. {pull}23417[23417] +- Add check for empty values in azure module. {pull}24156[24156] +- Change the `event.created` in Netflow events to be the time the event was created by Filebeat +- Fix Zoom module parameters for basic auth and url path. {pull}23779[23779] +- Use rfc6587 framing for fortinet firewall and clientendpoint filesets when transferring over tcp. {pull}23837[23837] +- Fix httpjson input logging so it doesn't conflict with ECS. {pull}23972[23972] +- Fix Logstash module handling of logstash.log.log_event.action field. {issue}20709[20709] +- aws/s3access dataset was populating event.duration using the wrong unit. {pull}23920[23920] +- Zoom module pipeline failed to ingest some chat_channel events. {pull}23904[23904] +- Fix Netlow module issue with missing `internal_networks` config parameter. {issue}24094[24094] {pull}24110[24110] +- in httpjson input using encode_as "application/x-www-form-urlencoded" now sets Content-Type correctly {issue}24331[24331] {pull}24336[24336] +- Fix default `scope` in `add_nomad_metadata`. {issue}24559[24559] + +*Heartbeat* + + +*Heartbeat* + + +*Journalbeat* + + +*Metricbeat* + +- Add stack monitoring section to elasticsearch module documentation {pull}#23286[23286] +- Fix ec2 metricset fields.yml and the integration test {pull}23726[23726] +- Unskip s3_request integration test. {pull}23887[23887] +- Add system.hostfs configuration option for system module. {pull}23831[23831] + +*Packetbeat* + + + +*Winlogbeat* + + +*Functionbeat* + +*Elastic Logging Plugin* + + +==== Added + +*Affecting all Beats* + +- Honor kube event resysncs to handle missed watch events {pull}22668[22668] +- Add autodiscover provider and metadata processor for Nomad. {pull}14954[14954] {pull}23324[23324] +- Add `processors.rate_limit.n.dropped` monitoring counter metric for the `rate_limit` processor. {pull}23330[23330] +- Deprecate aws_partition config parameter for AWS, use endpoint instead. {pull}23539[23539] +- Update the baseline version of Sarama (Kafka support library) to 1.27.2. {pull}23595[23595] +- Add kubernetes.volume.fs.used.pct field. {pull}23564[23564] +- Add the `enable_krb5_fast` flag to the Kafka output to explicitly opt-in to FAST authentication. {pull}23629[23629] +- Added new decode_xml processor to libbeat that is available to all beat types. {pull}23678[23678] +- Add deployment name in pod's meta. {pull}23610[23610] +- Added ECS 1.8 `host.os.type` field to `add_host_metadata` processor. {pull}23513[23513] +- Add `selector` information in kubernetes services' metadata. {pull}23730[23730] + +*Auditbeat* + +- Improve file_integrity monitoring when a file is created/deleted in quick succession. {issue}17347[17347] {pull}22170[22170] +- system/host: Add new ECS 1.8 field `os.type` in `host.os.type`. {pull}23513[23513] +- Update Auditbeat auditd module to ECS 1.8 {pull}23594[23594] {issue}23118[23118] + +*Filebeat* + + +- Add parsing of tcp flags to AWS vpcflow fileset {issue}228020[22820] {pull}23157[23157] +- Added support for first_event context in filebeat httpjson input {pull}23437[23437] +- Adding Threat Intel module {pull}21795[21795] +- Added username parsing from Cisco ASA message 302013. {pull}21196[21196] +- Added `encode_as` and `decode_as` options to httpjson along with pluggable encoders/decoders {pull}23478[23478] +- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by +- Added support for Cisco AMP API as a new fileset. {pull}22768[22768] +- Added RFC6587 framing option for tcp and unix inputs {issue}23663[23663] {pull}23724[23724] +- Added `application/x-ndjson` as decode option for httpjson input {pull}23521[23521] +- Added `application/x-www-form-urlencoded` as encode option for httpjson input {pull}23521[23521] +- Move aws-s3 input to GA. {pull}23631[23631] +- Populate `source.mac` and `destination.mac` for Suricata EVE events. {issue}23706[23706] {pull}23721[23721] +- Added string splitting for httpjson input {pull}24022[24022] +- Added Signatures fileset to Zeek module {pull}23772[23772] +- Upgrade Cisco ASA/FTD/Umbrella to ECS 1.8.0. {pull}23819[23819] +- Add new ECS user and categories features to google_workspace/gsuite {issue}23118[23118] {pull}23709[23709] +- Move crowdstrike JS processor to ingest pipelines and upgrade to ECS 1.8.0 {issue}23118[23118] {pull}23875[23875] +- Update Filebeat auditd dataset to ECS 1.8.0. {pull}23723[23723] {issue}23118[23118] +- Updated microsoft defender_atp and m365_defender to ECS 1.8. {pull}23897[23897] {issue}23118[23118] +- Updated o365 module to ECS 1.8. {issue}23118[23118] {pull}23896[23896] +- Upgrade CEF module to ECS 1.8.0. {pull}23832[23832] +- Upgrade fortinet/firewall to ECS 1.8 {issue}23118[23118] {pull}23902[23902] +- Upgrade Zeek to ECS 1.8.0. {issue}23118[23118] {pull}23847[23847] +- Updated azure module to ECS 1.8. {issue}23118[23118] {pull}23927[23927] +- Update aws/s3access to ECS 1.8. {issue}23118[23118] {pull}23920[23920] +- Upgrade panw module to ecs 1.8 {issue}23118[23118] {pull}23931[23931] +- Updated aws/cloudtrail fileset to ECS 1.8. {issue}23118[23118] {pull}23911[23911] +- Upgrade juniper/srx to ecs 1.8.0. {issue}23118[23118] {pull}23936[23936] +- Update mysqlenterprise module to ECS 1.8. {issue}23118[23118] {pull}23978[23978] +- Upgrade sophos/xg fileset to ECS 1.8.0. {issue}23118[23118] {pull}23967[23967] +- Upgrade system/auth to ECS 1.8 {issue}23118[23118] {pull}23961[23961] +- Upgrade elasticsearch/audit to ECS 1.8 {issue}23118[23118] {pull}24000[24000] +- Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] +- Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118] + +*Heartbeat* + +- Bundle synthetics deps with heartbeat docker image. {pull}23274[23274] + +*Heartbeat* + +- Update Journalbeat to ECS 1.8. {pull}23737[23737] + +*Heartbeat* + +*Journalbeat* + +*Metricbeat* + +- Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] +- Add support for Darwin/arm M1. {pull}24019[24019] +- Check fields are documented in aws metricsets. {pull}23887[23887] + +*Packetbeat* + + +- Upgrade to ECS 1.8.0. {pull}23783[23783] +- Add `event.type: [connection]` to flow events and include `end` for final flows. {pull}24564[24564] + +*Functionbeat* + +- Provide more ways to set AWS credentials. {issue}12464[12464] {pull}23344[23344] +- Add support for multiple regions {pull}21065[21065] + +*Heartbeat* + +- Add support for script processor. {pull}23229[23229] + +*Winlogbeat* + +- Add Audit and Authentication Polixy Change Events and related.ip information {pull}20684[20684] +- Add new ECS 1.8 improvements. {pull}23563[23563] +- Remove deprecated eventlogging api that was used for Windows XP/2003 and associated unused code. {pull}24463[24463] + +*Elastic Log Driver* + + +==== Deprecated + +*Affecting all Beats* + +- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as + +*Filebeat* + + +*Heartbeat* + +*Journalbeat* + +*Metricbeat* + + +*Packetbeat* + +*Winlogbeat* + +*Functionbeat* + +==== Known Issue + +*Journalbeat* + + + [[release-notes-7.11.2]] === Beats version 7.11.2 https://github.com/elastic/beats/compare/v7.11.1...v7.11.2[View commits] diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index a4c6df454a05..f62e4c2d9315 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -18,13 +18,11 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Remove `AddDockerMetadata` and `AddKubernetesMetadata` processors from the `script` processor. They can still be used as normal processors in the configuration. {issue}16349[16349] {pull}16514[16514] - Introduce APM libbeat instrumentation, active when running the beat with ELASTIC_APM_ACTIVE=true. {pull}17938[17938] - Make error message about locked data path actionable. {pull}18667[18667] -- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820] *Auditbeat* *Filebeat* -- Add fileset to ingest PostgreSQL CSV logs. {pull}23334[23334] - Fix parsing of Elasticsearch node name by `elasticsearch/slowlog` fileset. {pull}14547[14547] - Improve ECS field mappings in panw module. event.outcome now only contains success/failure per ECS specification. {issue}16025[16025] {pull}17910[17910] - Improve ECS categorization field mappings for nginx module. http.request.referrer only populated when nginx sets a value {issue}16174[16174] {pull}17844[17844] @@ -42,16 +40,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Disable the option of running --machine-learning on its own. {pull}20241[20241] - Fix PANW field spelling "veredict" to "verdict" on event.action {pull}18808[18808] - Add support for GMT timezone offsets in `decode_cef`. {pull}20993[20993] -- API address and shard ID are required settings in the Cloud Foundry input. {pull}21759[21759] -- Remove `suricata.eve.timestamp` alias field. {issue}10535[10535] {pull}22095[22095] -- Rename bad ECS field name tracing.trace.id to trace.id in aws elb fileset. {pull}22571[22571] -- Fix parsing issues with nested JSON payloads in Elasticsearch audit log fileset. {pull}22975[22975] -- Rename `network.direction` values in crowdstrike/falcon to `ingress`/`egress`. {pull}23041[23041] -- Rename `s3` input to `aws-s3` input. {pull}23469[23469] *Heartbeat* -- Adds negative body match. {pull}20728[20728] -- Refactor synthetics configuration to new syntax. {pull}23467[23467] *Journalbeat* @@ -64,13 +54,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - kubernetes.container.cpu.limit.cores and kubernetes.container.cpu.requests.cores are now floats. {issue}11975[11975] - Fix ECS compliance of user.id field in system/users metricset {pull}19019[19019] - Remove "invalid zero" metrics on Windows and Darwin, don't report linux-only memory and diskio metrics when running under agent. {pull}21457[21457] -- Change cloud.provider from googlecloud to gcp. {pull}21775[21775] -- API address and shard ID are required settings in the Cloud Foundry module. {pull}21759[21759] -- Rename googlecloud module to gcp module. {pull}22246[22246] -- Use ingress/egress instead of inbound/outbound for system/socket metricset. {pull}22992[22992] -- Change types of numeric metrics from Kubelet summary api to double so as to cover big numbers. {pull}23335[23335] -- Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] -- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}[23905] *Packetbeat* @@ -127,24 +110,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add service resource in k8s cluster role. {pull}20546[20546] - [Metricbeat][Kubernetes] Change cluster_ip field from ip to keyword. {pull}20571[20571] - The `o365input` and `o365` module now recover from an authentication problem or other fatal errors, instead of terminating. {pull}21258[21258] -- Orderly close processors when processing pipelines are not needed anymore to release their resources. {pull}16349[16349] -- Fix memory leak and events duplication in docker autodiscover and add_docker_metadata. {pull}21851[21851] -- Fix parsing of expired licences. {issue}21112[21112] {pull}22180[22180] -- Fix duplicated pod events in kubernetes autodiscover for pods with init or ephemeral containers. {pull}22438[22438] -- Fix FileVersion contained in Windows exe files. {pull}22581[22581] -- Fix index template loading when the new index format is selected. {issue}22482[22482] {pull}22682[22682] -- Log debug message if the Kibana dashboard can not be imported from the archive because of the invalid archive directory structure {issue}12211[12211], {pull}13387[13387] - Periodic metrics in logs will now report `libbeat.output.events.active` and `beat.memstats.rss` - as gauges (rather than counters). {pull}22877[22877] -- Use PROGRAMDATA environment variable instead of C:\ProgramData for windows install service {pull}22874[22874] -- Fix reporting of cgroup metrics when running under Docker {pull}22879[22879] -- Fix typo in config docs {pull}23185[23185] -- Fix `nested` subfield handling in generated Elasticsearch templates. {issue}23178[23178] {pull}23183[23183] -- Fix CPU usage metrics on VMs with dynamic CPU config {pull}23154[23154] -- Fix panic due to unhandled DeletedFinalStateUnknown in k8s OnDelete {pull}23419[23419] -- Fix error loop with runaway CPU use when the Kafka output encounters some connection errors {pull}23484[23484] -- Fix issue discovering docker containers and metadata after reconnections {pull}24318[24318] -- Allow configuring credential_profile_name and shared_credential_file when using role_arn. {pull}24174[24174] +as gauges (rather than counters). {pull}22877[22877] *Auditbeat* @@ -155,20 +122,9 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - system/package: Fix an error that can occur while trying to persist package metadata. {issue}18536[18536] {pull}18887[18887] - system/socket: Fix dataset using 100% CPU and becoming unresponsive in some scenarios. {pull}19033[19033] {pull}19764[19764] - system/socket: Fixed tracking of long-running connections. {pull}19033[19033] -- system/login: Fixed offset reset on inode reuse. {pull}24414[24414] -- system/login: Add additional offset check for utmp files. {pull}24515[24515] *Filebeat* -- CheckPoint Firewall module: Change event.severity JSON data type to a number because the field mapping is a `long`. {pull}23424[23424] -- Cisco IOS: Change icmp.type/code and igmp.type JSON data types to strings because the fields mappings are `keyword`. {pull}23424[23424] -- CrowdStrike Falcon: Change JSON field types to match the field mappings. {pull}23424[23424] -- Fortinet Firewall: Drop `fortinet.firewall.assignip` when the value is "N/A". {pull}23424[23424] -- Juniper SRX: Change JSON field types to match the field mappings. {pull}23424[23424] -- Suricata EVE: Convert `suricata.eve.flow_id` to string because the field is a keyword in the mapping. {pull}23424[23424] -- Zeek DNS: Ignore failures in data type conversions. And change `dns.id` JSON field to a string to match its `keyword` mapping. {pull}23424[23424] -- Update `filestream` reader offset when a line is skipped. {pull}23417[23417] -- Add check for empty values in azure module. {pull}24156[24156] - cisco/asa fileset: Fix parsing of 302021 message code. {pull}14519[14519] - Fix filebeat azure dashboards, event category should be `Alert`. {pull}14668[14668] - Fixed dashboard for Cisco ASA Firewall. {issue}15420[15420] {pull}15553[15553] @@ -211,46 +167,9 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Fix `cisco` asa and ftd parsing of messages 106102 and 106103. {pull}20469[20469] - Fix event.kind for system/syslog pipeline {issue}20365[20365] {pull}20390[20390] - Fix event.type for zeek/ssl and duplicate event.category for zeek/connection {pull}20696[20696] -- Fix long registry migration times. {pull}20717[20717] {issue}20705[20705] -- Fix event types and categories in auditd module to comply with ECS {pull}20652[20652] -- Update documentation in the azure module filebeat. {pull}20815[20815] -- Remove wrongly mapped `tls.client.server_name` from `fortinet/firewall` fileset. {pull}20983[20983] -- Fix an error updating file size being logged when EOF is reached. {pull}21048[21048] -- Fix error when processing AWS Cloudtrail Digest logs. {pull}21086[21086] {issue}20943[20943] -- Provide backwards compatibility for the `set` processor when Elasticsearch is less than 7.9.0. {pull}20908[20908] -- Handle multiple upstreams in ingress-controller. {pull}21215[21215] -- Provide backwards compatibility for the `append` processor when Elasticsearch is less than 7.10.0. {pull}21159[21159] -- Fix checkpoint module when logs contain time field. {pull}20567[20567] -- Add field limit check for AWS Cloudtrail flattened fields. {pull}21388[21388] {issue}21382[21382] -- Fix syslog RFC 5424 parsing in the CheckPoint module. {pull}21854[21854] - Add json body check for sqs message. {pull}21727[21727] -- Fix incorrect connection state mapping in zeek connection pipeline. {pull}22151[22151] {issue}22149[22149] -- Fix Zeek dashboard reference to `zeek.ssl.server.name` field. {pull}21696[21696] -- Fix handing missing eventtime and assignip field being set to N/A for fortinet module. {pull}22361[22361] -- Fix for `field [source] not present as part of path [source.ip]` error in azure pipelines. {pull}22377[22377] - Drop aws.vpcflow.pkt_srcaddr and aws.vpcflow.pkt_dstaddr when equal to "-". {pull}22721[22721] {issue}22716[22716] -- Fix cisco umbrella module config by adding input variable. {pull}22892[22892] -- Fix network.direction logic in zeek connection fileset. {pull}22967[22967] -- Fix aws s3 overview dashboard. {pull}23045[23045] -- Fix bad `network.direction` values in Fortinet/firewall fileset. {pull}23072[23072] -- Fix Cisco ASA/FTD module's parsing of WebVPN log message 716002. {pull}22966[22966] -- Add support for organization and custom prefix in AWS/CloudTrail fileset. {issue}23109[23109] {pull}23126[23126] -- Simplify regex for organization custom prefix in AWS/CloudTrail fileset. {issue}23203[23203] {pull}23204[23204] -- Fix syslog header parsing in infoblox module. {issue}23272[23272] {pull}23273[23273] -- Fix CredentialsJSON unpacking for `gcp-pubsub` and `httpjson` inputs. {pull}23277[23277] -- Fix concurrent modification exception in Suricata ingest node pipeline. {pull}23534[23534] -- Change the `event.created` in Netflow events to be the time the event was created by Filebeat - to be consistent with ECS. {pull}23094[23094] -- Fix Zoom module parameters for basic auth and url path. {pull}23779[23779] -- Fix handling of ModifiedProperties field in Office 365. {pull}23777[23777] -- Use rfc6587 framing for fortinet firewall and clientendpoint filesets when transferring over tcp. {pull}23837[23837] -- Fix httpjson input logging so it doesn't conflict with ECS. {pull}23972[23972] -- Fix Logstash module handling of logstash.log.log_event.action field. {issue}20709[20709] -- aws/s3access dataset was populating event.duration using the wrong unit. {pull}23920[23920] -- Zoom module pipeline failed to ingest some chat_channel events. {pull}23904[23904] -- Fix Netlow module issue with missing `internal_networks` config parameter. {issue}24094[24094] {pull}24110[24110] -- in httpjson input using encode_as "application/x-www-form-urlencoded" now sets Content-Type correctly {issue}24331[24331] {pull}24336[24336] -- Fix default `scope` in `add_nomad_metadata`. {issue}24559[24559] +to be consistent with ECS. {pull}23094[23094] *Heartbeat* @@ -313,20 +232,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add support for azure light metricset app_stats. {pull}20639[20639] - Fix remote_write flaky test. {pull}21173[21173] - Remove io.time from windows {pull}22237[22237] -- Change Session ID type from int to string {pull}22359[22359] -- Fix filesystem types on Windows in filesystem metricset. {pull}22531[22531] -- Fix failiures caused by custom beat names with more than 15 characters {pull}22550[22550] -- Stop generating NaN values from Cloud Foundry module to avoid errors in outputs. {pull}22634[22634] -- Update NATS dashboards to leverage connection and route metricsets {pull}22646[22646] -- Fix rate metrics in Kafka broker metricset by using last minute rate instead of mean rate. {pull}22733[22733] - Fix `logstash` module when `xpack.enabled: true` is set from emitting redundant events. {pull}22808[22808] -- Change vsphere.datastore.capacity.used.pct value to betweeen 0 and 1. {pull}23148[23148] -- Update config in `windows.yml` file. {issue}23027[23027]{pull}23327[23327] -- Add stack monitoring section to elasticsearch module documentation {pull}#23286[23286] -- Fix metric grouping for windows/perfmon module {issue}23489[23489] {pull}23505[23505] -- Fix ec2 metricset fields.yml and the integration test {pull}23726[23726] -- Unskip s3_request integration test. {pull}23887[23887] -- Add system.hostfs configuration option for system module. {pull}23831[23831] *Packetbeat* @@ -372,38 +278,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add capability of enriching process metadata with contianer id also for non-privileged containers in `add_process_metadata` processor. {pull}19767[19767] - Add replace_fields config option in add_host_metadata for replacing host fields. {pull}20490[20490] {issue}20464[20464] - Add option to select the type of index template to load: legacy, component, index. {pull}21212[21212] -- Add istiod metricset. {pull}21519[21519] -- Release `add_cloudfoundry_metadata` as GA. {pull}21525[21525] -- Add support for OpenStack SSL metadata APIs in `add_cloud_metadata`. {pull}21590[21590] -- Add cloud.account.id for GCP into add_cloud_metadata processor. {pull}21776[21776] -- Add proxy metricset for istio module. {pull}21751[21751] -- Add kubernetes.node.hostname metadata of Kubernetes node. {pull}22189[22189] -- Enable always add_resource_metadata for Pods and Services of kubernetes autodiscovery. {pull}22189[22189] -- Add add_resource_metadata option setting (always enabled) for add_kubernetes_metadata setting. {pull}22189[22189] -- Added Kafka version 2.2 to the list of supported versions. {pull}22328[22328] -- Add support for ephemeral containers in kubernetes autodiscover and `add_kubernetes_metadata`. {pull}22389[22389] {pull}22439[22439] -- Added support for wildcard fields and keyword fallback in beats setup commands. {pull}22521[22521] -- Fix polling node when it is not ready and monitor by hostname {pull}22666[22666] -- Add `expand_keys` option to `decode_json_fields` processor and `json` input, to recusively de-dot and expand json keys into hierarchical object structures {pull}22849[22849] -- Update k8s client and release k8s leader lock gracefully {pull}22919[22919] -- Improve event normalization performance {pull}22974[22974] -- Add tini as init system in docker images {pull}22137[22137] -- Added "detect_mime_type" processor for detecting mime types {pull}22940[22940] -- Added "add_network_direction" processor for determining perimeter-based network direction. {pull}23076[23076] -- Added new `rate_limit` processor for enforcing rate limits on event throughput. {pull}22883[22883] -- Allow node/namespace metadata to be disabled on kubernetes metagen and ensure add_kubernetes_metadata honors host {pull}23012[23012] -- Improve equals check. {pull}22778[22778] -- Honor kube event resysncs to handle missed watch events {pull}22668[22668] -- Add autodiscover provider and metadata processor for Nomad. {pull}14954[14954] {pull}23324[23324] -- Add `processors.rate_limit.n.dropped` monitoring counter metric for the `rate_limit` processor. {pull}23330[23330] -- Deprecate aws_partition config parameter for AWS, use endpoint instead. {pull}23539[23539] -- Update the baseline version of Sarama (Kafka support library) to 1.27.2. {pull}23595[23595] -- Add kubernetes.volume.fs.used.pct field. {pull}23564[23564] -- Add the `enable_krb5_fast` flag to the Kafka output to explicitly opt-in to FAST authentication. {pull}23629[23629] -- Added new decode_xml processor to libbeat that is available to all beat types. {pull}23678[23678] -- Add deployment name in pod's meta. {pull}23610[23610] -- Added ECS 1.8 `host.os.type` field to `add_host_metadata` processor. {pull}23513[23513] -- Add `selector` information in kubernetes services' metadata. {pull}23730[23730] *Auditbeat* @@ -412,11 +286,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Log to stderr when running using reference kubernetes manifests. {pull}17443[174443] - Fix syscall kprobe arguments for 32-bit systems in socket module. {pull}17500[17500] - Add ECS categorization info for auditd module {pull}18596[18596] -- Add several improvements for auditd module for improved ECS field mapping {pull}22647[22647] -- Add ECS 1.7 `configuration` categorization in certain events in auditd module. {pull}23000[23000] -- Improve file_integrity monitoring when a file is created/deleted in quick succession. {issue}17347[17347] {pull}22170[22170] -- system/host: Add new ECS 1.8 field `os.type` in `host.os.type`. {pull}23513[23513] -- Update Auditbeat auditd module to ECS 1.8 {pull}23594[23594] {issue}23118[23118] *Filebeat* @@ -497,97 +366,14 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add related.hosts ecs field to all modules {pull}21160[21160] - Keep cursor state between httpjson input restarts {pull}20751[20751] - New juniper.srx dataset for Juniper SRX logs. {pull}20017[20017] -- Adding support for Microsoft 365 Defender (Microsoft Threat Protection) {pull}21446[21446] -- Adding support for FIPS in s3 input {pull}21446[21446] -- Adding support for Oracle Database Audit Logs {pull}21991[21991] -- Add max_number_of_messages config into s3 input. {pull}21993[21993] -- Update Okta documentation for new stateful restarts. {pull}22091[22091] -- Add SSL option to checkpoint module {pull}19560[19560] -- Added support for MySQL Enterprise audit logs. {pull}22273[22273] -- Rename googlecloud module to gcp module. {pull}22214[22214] -- Rename awscloudwatch input to aws-cloudwatch. {pull}22228[22228] -- Rename google-pubsub input to gcp-pubsub. {pull}22213[22213] -- Copy tag names from MISP data into events. {pull}21664[21664] - Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291] -- Added TLS JA3 fingerprint, certificate not_before/not_after, certificate SHA1 hash, and certificate subject fields to Zeek SSL dataset. {pull}21696[21696] -- Add platform logs in the azure filebeat module. {pull}22371[22371] -- Added `event.ingested` field to data from the Netflow module. {pull}22412[22412] -- Improve panw ECS url fields mapping. {pull}22481[22481] -- Improve Nats filebeat dashboard. {pull}22726[22726] -- Add support for UNIX datagram sockets in `unix` input. {issues}18632[18632] {pull}22699[22699] -- Add `http.request.mime_type` for Elasticsearch audit log fileset. {pull}22975[22975] -- Add new httpjson input features and mark old config ones for deprecation {pull}22320[22320] -- Add configuration option to set external and internal networks for panw panos fileset {pull}22998[22998] -- Add `subbdomain` fields for rsa2elk modules. {pull}23035[23035] -- Add subdomain enrichment for suricata/eve fileset. {pull}23011[23011] -- Add subdomain enrichment for zeek/dns fileset. {pull}23011[23011] -- Add `event.category` "configuration" to auditd module events. {pull}23010[23010] -- Add `event.category` "configuration" to gsuite module events. {pull}23010[23010] -- Add `event.category` "configuration" to o365 module events. {pull}23010[23010] -- Add `event.category` "configuration" to zoom module events. {pull}23010[23010] -- Add `network.direction` to auditd/log fileset. {pull}23041[23041] -- Add logic for external network.direction in sophos xg fileset {pull}22973[22973] -- Preserve AWS CloudTrail eventCategory in aws.cloudtrail.event_category. {issue}22776[22776] {pull}22805[22805] -- Add top_level_domain enrichment for suricata/eve fileset. {pull}23046[23046] -- Add top_level_domain enrichment for zeek/dns fileset. {pull}23046[23046] -- Add `observer.egress.zone` and `observer.ingress.zone` for cisco/asa and cisco/ftd filesets. {pull}23068[23068] -- Allow cisco/asa and cisco/ftd filesets to override network directionality based off of zones. {pull}23068[23068] -- Allow cef and checkpoint modules to override network directionality based off of zones {pull}23066[23066] -- Add `network.direction` to netflow/log fileset. {pull}23052[23052] -- Add the ability to override `network.direction` based on interfaces in Fortinet/firewall fileset. {pull}23072[23072] -- Add `network.direction` override by specifying `internal_networks` in gcp module. {pull}23081[23081] -- Migrate microsoft/defender_atp to httpjson v2 config {pull}23017[23017] -- Migrate microsoft/m365_defender to httpjson v2 config {pull}23018[23018] -- Migrate okta to httpjson v2 config {pull}23059[23059] -- Add support for Snyk Vulnerability and Audit API. {pull}22677[22677] -- Misp improvements: Migration to httpjson v2 config, pagination and deduplication ID {pull}23070[23070] -- Add Google Workspace module and mark Gsuite module as deprecated {pull}22950[22950] -- Mark m365 defender, defender atp, okta and google workspace modules as GA {pull}23113[23113] -- Add parsing of tcp flags to AWS vpcflow fileset {issue}228020[22820] {pull}23157[23157] -- Added support for first_event context in filebeat httpjson input {pull}23437[23437] -- Added `alternative_host` option to google pubsub input {pull}23215[23215] -- Adding Threat Intel module {pull}21795[21795] -- Added username parsing from Cisco ASA message 302013. {pull}21196[21196] -- Added `encode_as` and `decode_as` options to httpjson along with pluggable encoders/decoders {pull}23478[23478] -- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by - removing unsupported processors. {pull}23763[23763] -- Added support for Cisco AMP API as a new fileset. {pull}22768[22768] -- Added RFC6587 framing option for tcp and unix inputs {issue}23663[23663] {pull}23724[23724] -- Added `application/x-ndjson` as decode option for httpjson input {pull}23521[23521] -- Added `application/x-www-form-urlencoded` as encode option for httpjson input {pull}23521[23521] -- Move aws-s3 input to GA. {pull}23631[23631] -- Populate `source.mac` and `destination.mac` for Suricata EVE events. {issue}23706[23706] {pull}23721[23721] -- Added string splitting for httpjson input {pull}24022[24022] -- Added Signatures fileset to Zeek module {pull}23772[23772] -- Upgrade Cisco ASA/FTD/Umbrella to ECS 1.8.0. {pull}23819[23819] -- Add new ECS user and categories features to google_workspace/gsuite {issue}23118[23118] {pull}23709[23709] -- Move crowdstrike JS processor to ingest pipelines and upgrade to ECS 1.8.0 {issue}23118[23118] {pull}23875[23875] -- Update Filebeat auditd dataset to ECS 1.8.0. {pull}23723[23723] {issue}23118[23118] -- Updated microsoft defender_atp and m365_defender to ECS 1.8. {pull}23897[23897] {issue}23118[23118] -- Updated o365 module to ECS 1.8. {issue}23118[23118] {pull}23896[23896] -- Upgrade CEF module to ECS 1.8.0. {pull}23832[23832] -- Upgrade fortinet/firewall to ECS 1.8 {issue}23118[23118] {pull}23902[23902] -- Upgrade Zeek to ECS 1.8.0. {issue}23118[23118] {pull}23847[23847] -- Updated azure module to ECS 1.8. {issue}23118[23118] {pull}23927[23927] -- Update aws/s3access to ECS 1.8. {issue}23118[23118] {pull}23920[23920] -- Upgrade panw module to ecs 1.8 {issue}23118[23118] {pull}23931[23931] -- Updated aws/cloudtrail fileset to ECS 1.8. {issue}23118[23118] {pull}23911[23911] -- Upgrade juniper/srx to ecs 1.8.0. {issue}23118[23118] {pull}23936[23936] -- Update mysqlenterprise module to ECS 1.8. {issue}23118[23118] {pull}23978[23978] -- Upgrade sophos/xg fileset to ECS 1.8.0. {issue}23118[23118] {pull}23967[23967] -- Upgrade system/auth to ECS 1.8 {issue}23118[23118] {pull}23961[23961] -- Upgrade elasticsearch/audit to ECS 1.8 {issue}23118[23118] {pull}24000[24000] -- Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] -- Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118] +removing unsupported processors. {pull}23763[23763] *Heartbeat* -- Add mime type detection for http responses. {pull}22976[22976] -- Bundle synthetics deps with heartbeat docker image. {pull}23274[23274] *Heartbeat* -- Update Journalbeat to ECS 1.8. {pull}23737[23737] *Heartbeat* @@ -646,54 +432,22 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Add billing metricset into googlecloud module. {pull}20812[20812] {issue}20738[20738] - Release lambda metricset in aws module as GA. {issue}21251[21251] {pull}21255[21255] - Add dashboard for pubsub metricset in googlecloud module. {pull}21326[21326] {issue}17137[17137] -- Move Prometheus query & remote_write to GA. {pull}21507[21507] -- Map cloud data filed `cloud.account.id` to azure subscription. {pull}21483[21483] {issue}21381[21381] -- Expand unsupported option from namespace to metrics in the azure module. {pull}21486[21486] -- Move s3_daily_storage and s3_request metricsets to use cloudwatch input. {pull}21703[21703] -- Duplicate system.process.cmdline field with process.command_line ECS field name. {pull}22325[22325] -- Add awsfargate module task_stats metricset to monitor AWS ECS Fargate. {pull}22034[22034] -- Add connection and route metricsets for nats metricbeat module to collect metrics per connection/route. {pull}22445[22445] -- Add unit file states to system/service {pull}22557[22557] -- `kibana` module: `stats` metricset no-longer collects usage-related data. {pull}22732[22732] -- Add more TCP states to Metricbeat system socket_summary. {pull}14347[14347] -- Add io.ops in fields exported by system.diskio. {pull}22066[22066] -- Adjust the Apache status fields in the fleet mode. {pull}22821[22821] -- Add AWS Fargate overview dashboard. {pull}22941[22941] -- Add process.state, process.cpu.pct, process.cpu.start_time and process.memory.pct. {pull}22845[22845] -- Move IIS module to GA and map fields. {issue}22609[22609] {pull}23024[23024] -- Apache: convert status.total_kbytes to status.total_bytes in fleet mode. {pull}23022[23022] -- Release MSSQL as GA {pull}23146[23146] -- Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] -- Add support for Darwin/arm M1. {pull}24019[24019] -- Check fields are documented in aws metricsets. {pull}23887[23887] *Packetbeat* -- Upgrade to ECS 1.8.0. {pull}23783[23783] -- Add `event.type: [connection]` to flow events and include `end` for final flows. {pull}24564[24564] *Functionbeat* -- Provide more ways to set AWS credentials. {issue}12464[12464] {pull}23344[23344] -- Add support for multiple regions {pull}21065[21065] *Heartbeat* -- Add support for script processor. {pull}23229[23229] *Winlogbeat* - Set process.command_line and process.parent.command_line from Sysmon Event ID 1. {pull}17327[17327] - Add support for event IDs 4673,4674,4697,4698,4699,4700,4701,4702,4768,4769,4770,4771,4776,4778,4779,4964 to the Security module {pull}17517[17517] - Add registry and code signature information and ECS categorization fields for sysmon module {pull}18058[18058] -- Add file.pe and process.pe fields to ProcessCreate & LoadImage events in Sysmon module. {issue}17335[17335] {pull}22217[22217] -- Add dns.question.subdomain fields for sysmon DNS events. {pull}22999[22999] -- Add additional event categorization for security and sysmon modules. {pull}22988[22988] -- Add dns.question.top_level_domain fields for sysmon DNS events. {pull}23046[23046] -- Add Audit and Authentication Polixy Change Events and related.ip information {pull}20684[20684] -- Add new ECS 1.8 improvements. {pull}23563[23563] -- Remove deprecated eventlogging api that was used for Windows XP/2003 and associated unused code. {pull}24463[24463] *Elastic Log Driver* @@ -703,10 +457,9 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d *Affecting all Beats* -- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as - a hostname when Subject Alternative Name is not present from v8.0. - Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new - major version of Beats. +a hostname when Subject Alternative Name is not present from v8.0. +Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new +major version of Beats. *Filebeat* @@ -727,3 +480,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d ==== Known Issue *Journalbeat* + + + diff --git a/libbeat/docs/release.asciidoc b/libbeat/docs/release.asciidoc index 7c6f3aa2a164..a53bf859bc3f 100644 --- a/libbeat/docs/release.asciidoc +++ b/libbeat/docs/release.asciidoc @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read <> for more detail about changes that affect upgrade. +* <> * <> * <> * <> From 8f3986241e831e7c70181c43aae9fc50e35befe4 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Fri, 19 Mar 2021 17:46:31 +0100 Subject: [PATCH 02/10] 23334 is not a breaking change --- CHANGELOG.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index e3a3bdee26f2..ad1cadd12fa7 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -17,7 +17,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Filebeat* -- Add fileset to ingest PostgreSQL CSV logs. {pull}23334[23334] - Rename `s3` input to `aws-s3` input. {pull}23469[23469] *Heartbeat* @@ -164,6 +163,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Upgrade elasticsearch/audit to ECS 1.8 {issue}23118[23118] {pull}24000[24000] - Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] - Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118] +- Add fileset to ingest PostgreSQL CSV logs. {pull}23334[23334] *Heartbeat* From b33fc085b2a5473f335bc191877a811c1e6ccfc5 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Fri, 19 Mar 2021 17:51:22 +0100 Subject: [PATCH 03/10] Fix link --- CHANGELOG.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index ad1cadd12fa7..06575ad2daad 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -29,7 +29,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Metricbeat* - Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] -- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}[23905] +- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}23905[23905] *Packetbeat* From 2d2add8c5def71d1d13d040545d012e022761365 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Fri, 19 Mar 2021 17:53:24 +0100 Subject: [PATCH 04/10] Fix dangling line --- CHANGELOG.next.asciidoc | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index f62e4c2d9315..dfbce6203cf8 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -111,7 +111,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - [Metricbeat][Kubernetes] Change cluster_ip field from ip to keyword. {pull}20571[20571] - The `o365input` and `o365` module now recover from an authentication problem or other fatal errors, instead of terminating. {pull}21258[21258] - Periodic metrics in logs will now report `libbeat.output.events.active` and `beat.memstats.rss` -as gauges (rather than counters). {pull}22877[22877] *Auditbeat* From a063eb8121c1f766b5ce03056b7b0028bb883702 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Fri, 19 Mar 2021 18:02:12 +0100 Subject: [PATCH 05/10] More fixes --- CHANGELOG.asciidoc | 2 +- CHANGELOG.next.asciidoc | 5 +---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 06575ad2daad..7a7e4e776c8c 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -134,7 +134,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Adding Threat Intel module {pull}21795[21795] - Added username parsing from Cisco ASA message 302013. {pull}21196[21196] - Added `encode_as` and `decode_as` options to httpjson along with pluggable encoders/decoders {pull}23478[23478] -- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by +- Added feature to modules to adapt Ingest Node pipelines for compatibility with older Elasticsearch versions by removing unsupported processors. {pull}23763[23763] - Added support for Cisco AMP API as a new fileset. {pull}22768[22768] - Added RFC6587 framing option for tcp and unix inputs {issue}23663[23663] {pull}23724[23724] - Added `application/x-ndjson` as decode option for httpjson input {pull}23521[23521] diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index dfbce6203cf8..a80b0e6807b9 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -366,7 +366,6 @@ to be consistent with ECS. {pull}23094[23094] - Keep cursor state between httpjson input restarts {pull}20751[20751] - New juniper.srx dataset for Juniper SRX logs. {pull}20017[20017] - Added DNS response IP addresses to `related.ip` in Suricata module. {pull}22291[22291] -removing unsupported processors. {pull}23763[23763] *Heartbeat* @@ -456,9 +455,7 @@ removing unsupported processors. {pull}23763[23763] *Affecting all Beats* -a hostname when Subject Alternative Name is not present from v8.0. -Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new -major version of Beats. +- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as a hostname when Subject Alternative Name is not present from v8.0. Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new major version of Beats. *Filebeat* From c3af412970a7ef9006f9e69d26ad75900bd9bcfb Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Fri, 19 Mar 2021 18:03:08 +0100 Subject: [PATCH 06/10] No need to add empty lines --- CHANGELOG.next.asciidoc | 3 --- 1 file changed, 3 deletions(-) diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index a80b0e6807b9..978379bc1834 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -476,6 +476,3 @@ to be consistent with ECS. {pull}23094[23094] ==== Known Issue *Journalbeat* - - - From 1aca6fc6f3cdccd369f2fd0628c00fa828aafcf1 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Mon, 22 Mar 2021 16:22:09 +0100 Subject: [PATCH 07/10] Some cleanup --- CHANGELOG.asciidoc | 23 +++-------------------- 1 file changed, 3 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 7a7e4e776c8c..a721835a1165 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -13,32 +13,13 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820] -*Auditbeat* - *Filebeat* - Rename `s3` input to `aws-s3` input. {pull}23469[23469] *Heartbeat* -- Refactor synthetics configuration to new syntax. {pull}23467[23467] - -*Journalbeat* - - - -*Metricbeat* - -- Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] -- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}23905[23905] - -*Packetbeat* - - -*Winlogbeat* - - -*Functionbeat* +- Refactor synthetics configuration to new syntax. {pull}23467[23467] ==== Bugfixes @@ -182,6 +163,8 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] - Add support for Darwin/arm M1. {pull}24019[24019] - Check fields are documented in aws metricsets. {pull}23887[23887] +- Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] +- Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}23905[23905] *Packetbeat* From 51c8509027ade76bddd370a6efa425b3f69f0d20 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Mon, 22 Mar 2021 17:19:00 +0100 Subject: [PATCH 08/10] Final cleanup --- CHANGELOG.asciidoc | 59 ++--------------------------------------- CHANGELOG.next.asciidoc | 1 - 2 files changed, 2 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index a721835a1165..3942f79567a7 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -9,10 +9,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] ==== Breaking changes -*Affecting all Beats* - -- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {pull}23820[23820] - *Filebeat* - Rename `s3` input to `aws-s3` input. {pull}23469[23469] @@ -28,7 +24,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Fix `nested` subfield handling in generated Elasticsearch templates. {issue}23178[23178] {pull}23183[23183] - Fix CPU usage metrics on VMs with dynamic CPU config {pull}23154[23154] - Allow configuring credential_profile_name and shared_credential_file when using role_arn. {pull}24174[24174] - +- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {issue}23820[23820] {pull}23858[23858] *Auditbeat* @@ -57,15 +53,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - in httpjson input using encode_as "application/x-www-form-urlencoded" now sets Content-Type correctly {issue}24331[24331] {pull}24336[24336] - Fix default `scope` in `add_nomad_metadata`. {issue}24559[24559] -*Heartbeat* - - -*Heartbeat* - - -*Journalbeat* - - *Metricbeat* - Add stack monitoring section to elasticsearch module documentation {pull}#23286[23286] @@ -73,18 +60,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Unskip s3_request integration test. {pull}23887[23887] - Add system.hostfs configuration option for system module. {pull}23831[23831] -*Packetbeat* - - - -*Winlogbeat* - - -*Functionbeat* - -*Elastic Logging Plugin* - - ==== Added *Affecting all Beats* @@ -109,7 +84,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Filebeat* - - Add parsing of tcp flags to AWS vpcflow fileset {issue}228020[22820] {pull}23157[23157] - Added support for first_event context in filebeat httpjson input {pull}23437[23437] - Adding Threat Intel module {pull}21795[21795] @@ -154,10 +128,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Update Journalbeat to ECS 1.8. {pull}23737[23737] -*Heartbeat* - -*Journalbeat* - *Metricbeat* - Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] @@ -168,7 +138,6 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Packetbeat* - - Upgrade to ECS 1.8.0. {pull}23783[23783] - Add `event.type: [connection]` to flow events and include `end` for final flows. {pull}24564[24564] @@ -187,35 +156,11 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Add new ECS 1.8 improvements. {pull}23563[23563] - Remove deprecated eventlogging api that was used for Windows XP/2003 and associated unused code. {pull}24463[24463] -*Elastic Log Driver* - - ==== Deprecated *Affecting all Beats* -- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as - -*Filebeat* - - -*Heartbeat* - -*Journalbeat* - -*Metricbeat* - - -*Packetbeat* - -*Winlogbeat* - -*Functionbeat* - -==== Known Issue - -*Journalbeat* - +- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as a hostname when Subject Alternative Name is not present from v8.0. Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new major version of Beats. [[release-notes-7.11.2]] diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index 978379bc1834..722600e23561 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -455,7 +455,6 @@ to be consistent with ECS. {pull}23094[23094] *Affecting all Beats* -- Selecting `full` in `ssl.verification_mode` option will not treat CommonName field in x509 certificates as a hostname when Subject Alternative Name is not present from v8.0. Please update your certificates so it contains at least one DNSName instead of relying on CommonName in the new major version of Beats. *Filebeat* From 770a5cee7a65e555ec3d4ffd98a3dc7298a447c6 Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Mon, 22 Mar 2021 17:22:27 +0100 Subject: [PATCH 09/10] Additional fix --- CHANGELOG.next.asciidoc | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index d678bee11b8b..91fecb4957b4 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -175,7 +175,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d - Fix event.type for zeek/ssl and duplicate event.category for zeek/connection {pull}20696[20696] - Add json body check for sqs message. {pull}21727[21727] - Drop aws.vpcflow.pkt_srcaddr and aws.vpcflow.pkt_dstaddr when equal to "-". {pull}22721[22721] {issue}22716[22716] -to be consistent with ECS. {pull}23094[23094] - Fix cisco umbrella module config by adding input variable. {pull}22892[22892] - Fix network.direction logic in zeek connection fileset. {pull}22967[22967] - Fix aws s3 overview dashboard. {pull}23045[23045] From 912168d36e6f19dded5b42e46196d6f6a32f302b Mon Sep 17 00:00:00 2001 From: Andres Rodriguez Date: Mon, 22 Mar 2021 17:43:50 +0100 Subject: [PATCH 10/10] Apply suggestions from code review Co-authored-by: Brandon Morelli --- CHANGELOG.asciidoc | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 3942f79567a7..68d0f9291afd 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -24,7 +24,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Fix `nested` subfield handling in generated Elasticsearch templates. {issue}23178[23178] {pull}23183[23183] - Fix CPU usage metrics on VMs with dynamic CPU config {pull}23154[23154] - Allow configuring credential_profile_name and shared_credential_file when using role_arn. {pull}24174[24174] -- Fix panic with inline SSL when the certificate or key were small than 256 bytes. {issue}23820[23820] {pull}23858[23858] +- Fix panic with inline SSL when the certificate or key was smaller than 256 bytes. {issue}23820[23820] {pull}23858[23858] *Auditbeat* @@ -74,7 +74,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Added new decode_xml processor to libbeat that is available to all beat types. {pull}23678[23678] - Add deployment name in pod's meta. {pull}23610[23610] - Added ECS 1.8 `host.os.type` field to `add_host_metadata` processor. {pull}23513[23513] -- Add `selector` information in kubernetes services' metadata. {pull}23730[23730] +- Add `selector` information in Kubernetes services' metadata. {pull}23730[23730] *Auditbeat* @@ -85,7 +85,7 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Filebeat* - Add parsing of tcp flags to AWS vpcflow fileset {issue}228020[22820] {pull}23157[23157] -- Added support for first_event context in filebeat httpjson input {pull}23437[23437] +- Added support for first_event context in Filebeat httpjson input {pull}23437[23437] - Adding Threat Intel module {pull}21795[21795] - Added username parsing from Cisco ASA message 302013. {pull}21196[21196] - Added `encode_as` and `decode_as` options to httpjson along with pluggable encoders/decoders {pull}23478[23478] @@ -109,20 +109,20 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] - Upgrade Zeek to ECS 1.8.0. {issue}23118[23118] {pull}23847[23847] - Updated azure module to ECS 1.8. {issue}23118[23118] {pull}23927[23927] - Update aws/s3access to ECS 1.8. {issue}23118[23118] {pull}23920[23920] -- Upgrade panw module to ecs 1.8 {issue}23118[23118] {pull}23931[23931] +- Upgrade panw module to ECS 1.8 {issue}23118[23118] {pull}23931[23931] - Updated aws/cloudtrail fileset to ECS 1.8. {issue}23118[23118] {pull}23911[23911] -- Upgrade juniper/srx to ecs 1.8.0. {issue}23118[23118] {pull}23936[23936] +- Upgrade juniper/srx to ECS 1.8.0. {issue}23118[23118] {pull}23936[23936] - Update mysqlenterprise module to ECS 1.8. {issue}23118[23118] {pull}23978[23978] - Upgrade sophos/xg fileset to ECS 1.8.0. {issue}23118[23118] {pull}23967[23967] - Upgrade system/auth to ECS 1.8 {issue}23118[23118] {pull}23961[23961] - Upgrade elasticsearch/audit to ECS 1.8 {issue}23118[23118] {pull}24000[24000] -- Upgrade okta to ecs 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] +- Upgrade okta to ECS 1.8.0 and move js processor to ingest pipeline {issue}23118[23118] {pull}23929[23929] - Update zoom module to ECS 1.8. {pull}23904[23904] {issue}23118[23118] - Add fileset to ingest PostgreSQL CSV logs. {pull}23334[23334] *Heartbeat* -- Bundle synthetics deps with heartbeat docker image. {pull}23274[23274] +- Bundle synthetics dependencies with Heartbeat docker image. {pull}23274[23274] *Heartbeat* @@ -130,9 +130,9 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Metricbeat* -- Enrich events of `state_service` metricset with kubernetes services' metadata. {pull}23730[23730] +- Enrich events of `state_service` metricset with Kubernetes services' metadata. {pull}23730[23730] - Add support for Darwin/arm M1. {pull}24019[24019] -- Check fields are documented in aws metricsets. {pull}23887[23887] +- Check fields are documented in AWS metricsets. {pull}23887[23887] - Add container.image.name and containe.name ECS fields for state_container. {pull}23802[23802] - Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. {pull}23905[23905] @@ -152,9 +152,9 @@ https://github.com/elastic/beats/compare/v7.11.2...v7.12.0[View commits] *Winlogbeat* -- Add Audit and Authentication Polixy Change Events and related.ip information {pull}20684[20684] +- Add Audit and Authentication Policy Change Events and related.ip information {pull}20684[20684] - Add new ECS 1.8 improvements. {pull}23563[23563] -- Remove deprecated eventlogging api that was used for Windows XP/2003 and associated unused code. {pull}24463[24463] +- Remove deprecated eventlogging API that was used for Windows XP/2003 and associated unused code. {pull}24463[24463] ==== Deprecated