Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate ec2 metricset to use cloudwatch input as light weight module #25924

Merged
merged 8 commits into from
Jun 1, 2021
Merged

Migrate ec2 metricset to use cloudwatch input as light weight module #25924

merged 8 commits into from
Jun 1, 2021

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented May 26, 2021

What does this PR do?

This PR is to migrate ec2 metricset to use cloudwatch input as light weight module. Two main parts in this PR:

  1. In the original ec2 metricset, we are also making EC2 DescribeInstances API call to get metadata for each EC2 instance. This is added as a part of cloudwatch metricset now.
    You can see under ec2 metricset, there is only a manifest.yml file which defines cloudwatch as the input.

  2. We need to keep backwards compatibility with metric names so dashboard can still be used.
    You can see rename processor is used in manifest.yml file under ec2 metricset to make sure with this PR we are not introducing breaking changes. Also please check the updated data.json file in ec2 metricset to see the renaming is effective.

Why is it important?

With this change, we no longer need to maintain the common code in both ec2 metricset and cloudwatch metricset.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2021
@kaiyan-sheng kaiyan-sheng self-assigned this May 26, 2021
@kaiyan-sheng kaiyan-sheng added the Team:Integrations Label for the Integrations team label May 26, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2021
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 26, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #25924 updated

  • Start Time: 2021-05-31T14:22:32.378+0000

  • Duration: 78 min 22 sec

  • Commit: 43ec3f0

Test stats 🧪

Test Results
Failed 0
Passed 2033
Skipped 192
Total 2225

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 2033
Skipped 192
Total 2225

@kaiyan-sheng
Copy link
Contributor Author

CI failure is also reported in auditbeat. It doesn't look like it's related to this PR: #25956

@mergify
Copy link
Contributor

mergify bot commented May 28, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b refactor_cloudwatch_input upstream/refactor_cloudwatch_input
git merge upstream/master
git push upstream refactor_cloudwatch_input

@mtojek
Copy link
Contributor

mtojek commented May 28, 2021

/test

Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see this cleanup! Left some suggestions/comments but overall I'm ok with the change. Let me know what you think!

x-pack/metricbeat/module/aws/cloudwatch/ec2/metadata.go Outdated Show resolved Hide resolved
x-pack/metricbeat/module/aws/cloudwatch/metadata.go Outdated Show resolved Hide resolved
x-pack/metricbeat/module/aws/ec2/ec2_test.go Show resolved Hide resolved
@kaiyan-sheng kaiyan-sheng added the needs_backport PR is waiting to be backported to other branches. label May 28, 2021
Copy link
Member

@ChrsMark ChrsMark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@kaiyan-sheng kaiyan-sheng merged commit 80bd6ae into elastic:master Jun 1, 2021
@kaiyan-sheng kaiyan-sheng deleted the refactor_cloudwatch_input branch June 1, 2021 13:48
@kaiyan-sheng kaiyan-sheng added v7.14.0 and removed needs_backport PR is waiting to be backported to other branches. labels Jun 1, 2021
kaiyan-sheng added a commit that referenced this pull request Jun 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Integrations Label for the Integrations team v7.14.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants