Skip to content

Commit

Permalink
Merge branch 'master' into 27608-fix-elastic-agent-build
Browse files Browse the repository at this point in the history 
* master: (39 commits)
  [Heartbeat] Move JSON tests from python->go (elastic#27816)
  docs: simplify permissions for Dockerfile COPY (elastic#27754)
  Osquerybeat: Fix osquery logger plugin severy levels mapping (elastic#27789)
  [Filebeat] Update compatibility function to remove processor description on ES < 7.9.0 (elastic#27774)
  warn log entry and no validation failure when both queue_url and buck… (elastic#27612)
  libbeat/cmd/instance: ensure test config file has appropriate permissions (elastic#27178)
  [Heartbeat] Add httpcommon options to ZipURL (elastic#27699)
  Add a header round tripper option to httpcommon (elastic#27509)
  [Elastic Agent] Add validation to ensure certificate paths are absolute. (elastic#27779)
  Rename dashboards according to module.yml files for master (elastic#27749)
  Refactor vagrantfile, add scripts for provisioning with docker/kind (elastic#27726)
  Accept syslog dates with leading 0 (elastic#27775)
  [Filebeat] Add timezone config option to decode_cef and syslog input (elastic#27727)
  [Filebeat] Threatintel compatibility updates (elastic#27323)
  Add support for ephemeral containers in elastic agent dynamic provider (elastic#27707)
  [Filebeat] Integration tests in CI for AWS-S3 input (elastic#27491)
  Fix flakyness of TestFilestreamEmptyLine (elastic#27705)
  [Filebeat] kafka v2 using parsers (elastic#27335)
  Update Kafka version parsing / supported range (elastic#27720)
  Update Sarama to 1.29.1 (elastic#27717)
  ...
  • Loading branch information
@mdelapenya
mdelapenya committed Sep 9, 2021
2 parents 2337d09 + b04ac92 commit 16ae989
Show file tree
Hide file tree
Showing 221 changed files with 11,537 additions and 4,949 deletions.
14 changes: 14 additions & 0 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add daemonset.name in pods controlled by DaemonSets {pull}26808[26808], {issue}25816[25816]
- Kubernetes autodiscover fails in node scope if node name cannot be discovered {pull}26947[26947]
- Loading Kibana assets (dashboards, index templates) rely on Saved Object API. So to provide a reliable service, Beats can only import and export dasbhboards using at least Kibana 7.15. {issue}20672[20672] {pull}27220[27220]
- Skip add_kubernetes_metadata processor when kubernetes metadata are already present {pull}27689[27689]

*Auditbeat*

Expand Down Expand Up @@ -82,6 +83,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- threatintel module: Changed the type of `threatintel.indicator.first_seen` from `keyword` to `date`. {pull}26765[26765]
- Remove all alias fields pointing to ECS fields from modules. This affects the Suricata and Traefik modules. {issue}10535[10535] {pull}26627[26627]
- Add option for S3 input to work without SQS notification {issue}18205[18205] {pull}27332[27332]
- Fix Crowdstrike ingest pipeline that was creating flattened `process` fields. {issue}27622[27622] {pull}27623[27623]

*Heartbeat*
- Remove long deprecated `watch_poll` functionality. {pull}27166[27166]
Expand All @@ -105,6 +107,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Bump AWS SDK version to v0.24.0 for WebIdentity authentication flow {issue}19393[19393] {pull}27126[27126]
- Add Linux pressure metricset {pull}27355[27355]
- Add support for kube-state-metrics v2.0.0 {pull}27552[27552]
- Add User-Agent header to HTTP requests. {issue}18160[18160] {pull}27509[27509]

*Packetbeat*

Expand Down Expand Up @@ -204,6 +207,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Do not try to load ILM policy if `check_exists` is `false`. {pull}27508[27508] {issue}26322[26322]
- Fix bug with cgroups hierarchy override path in cgroups {pull}27620[27620]
- Beat `setup kibana` command may use the elasticsearch API key defined in `output.elasticsearch.api_key`. {issue}24015[24015] {pull}27540[27540]
- Seperate namespaces for V1 and V2 controller paths {pull}27676[27676]

*Auditbeat*

Expand Down Expand Up @@ -305,6 +309,9 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fixes a bug in `http_endpoint` that caused numbers encoded as strings. {issue}27382[27382] {pull}27480[27480]
- Update indentation for azure filebeat configuration. {pull}26604[26604]
- Auditd: Fix Top Exec Commands dashboard visualization. {pull}27638[27638]
- Store offset in `log.offset` field of events from the filestream input. {pull}27688[27688]
- Fix `httpjson` input rate limit processing and documentation. {pull}[]
- Update Filebeat compatibility function to remove processor description field on ES < 7.9.0 {pull}27774[27774]

*Heartbeat*

Expand Down Expand Up @@ -514,6 +521,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- update ECS field definitions to ECS 1.11.0. {pull}27107[27107]
- The disk queue is now GA. {pull}27515[27515]
- Allow non-padded base64 data to be decoded by decode_base64_field {pull}27311[27311], {issue}27021[27021]
- The Kafka support library Sarama has been updated to 1.29.1. {pull}27717[27717]
- Kafka is now supported up to version 2.8.0. {pull}27720[27720]

*Auditbeat*

Expand Down Expand Up @@ -730,9 +739,14 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add support to merge registry updates in the filestream input across multiple ACKed batches in case of backpressure in the registry or disk. {pull}25976[25976]
- Add support to `decode_cef` for MAC addresses that do not contain separator characters. {issue}27050[27050] {pull}27109[27109]
- Add new `hmac` template function for httpjson input {pull}27168[27168]
- Add `timezone` config option to the `decode_cef` processor. {issue}27232[27232] {pull}27727[27727]
- Add `timezone` config option to the `syslog` input. {pull}27727[27727]
- Update `tags` and `threatintel.indicator.provider` fields in `threatintel.anomali` ingest pipeline {issue}24746[24746] {pull}27141[27141]
- Move AWS module and filesets to GA. {pull}27428[27428]
- update ecs.version to ECS 1.11.0. {pull}27107[27107]
- Added support for parsing syslog dates containing a leading 0 (e.g. `Sep 01`) rather than a space. {pull}27775[27775]
- Add base64 Encode functionality to httpjson input. {pull}27681[27681]
- Add `join` and `sprintf` functions to `httpjson` input. {pull}27735[27735]


*Heartbeat*
Expand Down
27 changes: 16 additions & 11 deletions Jenkinsfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ pipeline {
RUNBLD_DISABLE_NOTIFICATIONS = 'true'
SLACK_CHANNEL = "#beats-build"
SNAPSHOT = 'true'
TERRAFORM_VERSION = "0.12.30"
TERRAFORM_VERSION = "0.13.7"
XPACK_MODULE_PATTERN = '^x-pack\\/[a-z0-9]+beat\\/module\\/([^\\/]+)\\/.*'
}
options {
Expand Down Expand Up @@ -277,14 +277,16 @@ def generateStages(Map args = [:]) {
}

def cloud(Map args = [:]) {
withNode(labels: args.label, forceWorkspace: true){
startCloudTestEnv(name: args.directory, dirs: args.dirs)
}
withCloudTestEnv() {
try {
target(context: args.context, command: args.command, directory: args.directory, label: args.label, withModule: args.withModule, isMage: true, id: args.id)
} finally {
terraformCleanup(name: args.directory, dir: args.directory)
withGithubNotify(context: args.context) {
withNode(labels: args.label, forceWorkspace: true){
startCloudTestEnv(name: args.directory, dirs: args.dirs)
}
withCloudTestEnv() {
try {
target(context: args.context, command: args.command, directory: args.directory, label: args.label, withModule: args.withModule, isMage: true, id: args.id)
} finally {
terraformCleanup(name: args.directory, dir: args.directory)
}
}
}
}
Expand Down Expand Up @@ -838,8 +840,10 @@ def withCloudTestEnv(Closure body) {
def maskedVars = []
def testTags = "${env.TEST_TAGS}"

// AWS
if (params.allCloudTests || params.awsCloudTests) {
// Allow AWS credentials when the build was configured to do so with:
// - the cloudtests build parameters
// - the aws github label
if (params.allCloudTests || params.awsCloudTests || matchesPrLabel(label: 'aws')) {
testTags = "${testTags},aws"
def aws = getVaultSecret(secret: "${AWS_ACCOUNT_SECRET}").data
if (!aws.containsKey('access_key')) {
Expand Down Expand Up @@ -891,6 +895,7 @@ def startCloudTestEnv(Map args = [:]) {
// If it failed then cleanup without failing the build
sh(label: 'Terraform Cleanup', script: ".ci/scripts/terraform-cleanup.sh ${folder}", returnStatus: true)
}
error('startCloudTestEnv: terraform apply failed.')
} finally {
// Archive terraform states in case manual cleanup is needed.
archiveArtifacts(allowEmptyArchive: true, artifacts: '**/terraform.tfstate')
Expand Down
Loading

0 comments on commit 16ae989

Please sign in to comment.