Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat] aws/vpcflow move processing to ingest node #28168

Merged
merged 2 commits into from
Sep 28, 2021

Conversation

leehinman
Copy link
Contributor

What does this PR do?

  • move aws/vpcflow processing to ingest node
  • sync with aws/vpcflow integration
  • make event.original optional
  • add preserve_original_event to default tags
  • check for ecs.version in config or pipeline

Why is it important?

  • Increases throughput of aws/vpcflow fileset
  • sync with aws/vpcflow integration
  • allows ecs.version to be in pipeline

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=aws TESTING_FILEBEAT_FILESETS=vpcflow mage -v pythonIntegTest

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 28, 2021
@mergify
Copy link
Contributor

mergify bot commented Sep 28, 2021

This pull request does not have a backport label. Could you fix it @leehinman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Sep 28, 2021
@leehinman leehinman added backport-v7.16.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify enhancement Filebeat Filebeat Team:Security-External Integrations labels Sep 28, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 28, 2021
- move aws/vpcflow processing to ingest node
- sync with aws/vpcflow integration
- make event.original optional
- add preserve_original_event to default tags
- check for ecs.version in config or pipeline
@leehinman leehinman removed the backport-skip Skip notification from the automated backport with mergify label Sep 28, 2021
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic 😄

LGTM

@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 28, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-09-28T19:32:20.557+0000

  • Duration: 148 min 1 sec

  • Commit: f1d41bd

Test stats 🧪

Test Results
Failed 0
Passed 54137
Skipped 5328
Total 59465

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

@leehinman leehinman merged commit 6e69b05 into elastic:master Sep 28, 2021
@leehinman leehinman deleted the vpcflow_to_ingest_pipeline branch September 28, 2021 22:19
mergify bot pushed a commit that referenced this pull request Sep 28, 2021
* [Filebeat] aws/vpcflow move processing to ingest node

- move aws/vpcflow processing to ingest node
- sync with aws/vpcflow integration
- make event.original optional
- add preserve_original_event to default tags
- check for ecs.version in config or pipeline

(cherry picked from commit 6e69b05)
mergify bot pushed a commit that referenced this pull request Sep 28, 2021
* [Filebeat] aws/vpcflow move processing to ingest node

- move aws/vpcflow processing to ingest node
- sync with aws/vpcflow integration
- make event.original optional
- add preserve_original_event to default tags
- check for ecs.version in config or pipeline

(cherry picked from commit 6e69b05)
leehinman added a commit that referenced this pull request Sep 29, 2021
* [Filebeat] aws/vpcflow move processing to ingest node

- move aws/vpcflow processing to ingest node
- sync with aws/vpcflow integration
- make event.original optional
- add preserve_original_event to default tags
- check for ecs.version in config or pipeline

(cherry picked from commit 6e69b05)

Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>
Icedroid pushed a commit to Icedroid/beats that referenced this pull request Nov 1, 2021
* [Filebeat] aws/vpcflow move processing to ingest node

- move aws/vpcflow processing to ingest node
- sync with aws/vpcflow integration
- make event.original optional
- add preserve_original_event to default tags
- check for ecs.version in config or pipeline
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v7.16.0 Automated backport with mergify backport-v8.0.0 Automated backport with mergify enhancement Filebeat Filebeat
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants