[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28285

aspacca · 2021-10-06T14:26:32Z

…t on aws

Enanchment

What does this PR do?

Fetches IMDSv2 token in add_cloud_metadata processor for AWS and send it with the proper header to the identity url request

Why is it important?

It adds support for IMDSv2 on AWS enforcing security standard

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
~~- [ ] I have added tests that prove my fix is effective or that my feature works~~
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

Run any beat built from this adding add_cloud_metadata processor an an EC2 instance with only IMDSv2 enabled and ensure that metadata request doesn't fail with a 401

Related issues

Closes #22101

Use cases

Screenshots

Logs

…t on aws

mergify · 2021-10-06T14:26:39Z

This pull request does not have a backport label. Could you fix it @aspacca? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2021-10-06T14:26:48Z

Pinging @elastic/integrations (Team:Integrations)

mergify · 2021-10-06T14:26:51Z

This pull request does not have a backport label. Could you fix it @aspacca? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

elasticmachine · 2021-10-06T15:30:32Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-10-13T08:35:50.189+0000
Duration: 149 min 56 sec
Commit: fc33e85

Test stats 🧪

Test	Results
Failed	0
Passed	53757
Skipped	5346
Total	59103

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.

mergify · 2021-10-11T15:19:11Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b Libbeat-Security-Enable-IMDSv2-support upstream/Libbeat-Security-Enable-IMDSv2-support
git merge upstream/master
git push upstream Libbeat-Security-Enable-IMDSv2-support

kaiyan-sheng

Overall looks good to me, just several small comments.

kaiyan-sheng · 2021-10-12T13:51:23Z

libbeat/processors/add_cloud_metadata/provider_aws_ec2.go

+const ec2InstanceIMDSv2TokenValueHeader = "X-aws-ec2-metadata-token"
+const ec2InstanceIMDSv2TokenTTLHeader = "X-aws-ec2-metadata-token-ttl-seconds"
+const ec2InstanceIMDSv2TokenTTLValue = "21600"
+const ec2InstanceIMDSv2TokenURI = "/latest/api/token"


Suggested change

const ec2InstanceIMDSv2TokenURI = "/latest/api/token"

const (

ec2InstanceIdentityURI = "/2014-02-25/dynamic/instance-identity/document"

ec2InstanceIMDSv2TokenURI = "/latest/api/token"

ec2InstanceIMDSv2TokenTTLHeader = "X-aws-ec2-metadata-token-ttl-seconds"

ec2InstanceIMDSv2TokenTTLValue = "21600"

ec2InstanceIMDSv2TokenURI = "/latest/api/token"

)

kaiyan-sheng · 2021-10-12T13:51:39Z

libbeat/processors/add_cloud_metadata/provider_aws_ec2.go

+	"net/http"
+
+	"github.com/elastic/beats/v7/libbeat/logp"
+


remove the empty space

kaiyan-sheng · 2021-10-12T13:52:50Z

libbeat/processors/add_cloud_metadata/provider_aws_ec2.go

+
+	tlsConfig, err := tlscommon.LoadTLSConfig(config.TLS)
+	if err != nil {
+		logger.Warnf("error while getting IMDSv2 token: %s. No token in the metadata request will be used.", err)


Suggested change

logger.Warnf("error while getting IMDSv2 token: %s. No token in the metadata request will be used.", err)

logger.Warnf("error when load TLS config for getting IMDSv2 token: %s. No token in the metadata request will be used.", err)

All the warning log messages are the same in the function. What do you think about adding more detail in the warning messages?

#28285) * [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata support on aws (cherry picked from commit 86decbd)

…d_cloud_metadata suppor… (#28400) * [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata support on aws (cherry picked from commit 86decbd)

elastic#28285) * [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata support on aws

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor…

0ec9307

…t on aws

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Oct 6, 2021

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Oct 6, 2021

aspacca added Team:Integrations Label for the Integrations team and removed backport-skip Skip notification from the automated backport with mergify labels Oct 6, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Oct 6, 2021

aspacca requested a review from kaiyan-sheng October 6, 2021 14:26

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Oct 6, 2021

aspacca added backport-v7.16.0 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify labels Oct 6, 2021

Andrea Spacca added 2 commits October 6, 2021 17:17

Merge branch 'master' into Libbeat-Security-Enable-IMDSv2-support

f437eeb

changelog

d894bfd

Merge branch 'master' into Libbeat-Security-Enable-IMDSv2-support

86be05b

kaiyan-sheng approved these changes Oct 12, 2021

View reviewed changes

Andrea Spacca added 2 commits October 13, 2021 10:03

Merge branch 'master' into Libbeat-Security-Enable-IMDSv2-support

5106c05

cr fixes

fc33e85

aspacca merged commit 86decbd into elastic:master Oct 13, 2021

mergify bot pushed a commit that referenced this pull request Oct 13, 2021

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… (

e8d9db8

#28285) * [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata support on aws (cherry picked from commit 86decbd)

mergify bot mentioned this pull request Oct 13, 2021

[7.x](backport #28285) [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28400

Merged

Icedroid pushed a commit to Icedroid/beats that referenced this pull request Nov 1, 2021

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… (

ed545c0

elastic#28285) * [Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata support on aws

andrewkroh mentioned this pull request Sep 12, 2022

[libbeat] add_cloud_metadata - startup blocked by AWS IMSDv2 token fetch #33058

Closed

constanca-m mentioned this pull request Oct 12, 2023

[add_cloud_metadata] Remove logger for AWS/EC2 #36829

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28285

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28285

aspacca commented Oct 6, 2021 •

edited

Loading

mergify bot commented Oct 6, 2021

elasticmachine commented Oct 6, 2021

mergify bot commented Oct 6, 2021

elasticmachine commented Oct 6, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

mergify bot commented Oct 11, 2021

kaiyan-sheng left a comment

kaiyan-sheng Oct 12, 2021

kaiyan-sheng Oct 12, 2021

kaiyan-sheng Oct 12, 2021

kaiyan-sheng Oct 12, 2021

	logger.Warnf("error while getting IMDSv2 token: %s. No token in the metadata request will be used.", err)
	logger.Warnf("error when load TLS config for getting IMDSv2 token: %s. No token in the metadata request will be used.", err)

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28285

[Libbeat] Security - fetch IMDSv2 token for add_cloud_metadata suppor… #28285

Conversation

aspacca commented Oct 6, 2021 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

mergify bot commented Oct 6, 2021

elasticmachine commented Oct 6, 2021

mergify bot commented Oct 6, 2021

elasticmachine commented Oct 6, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

mergify bot commented Oct 11, 2021

kaiyan-sheng left a comment

Choose a reason for hiding this comment

kaiyan-sheng Oct 12, 2021

Choose a reason for hiding this comment

kaiyan-sheng Oct 12, 2021

Choose a reason for hiding this comment

kaiyan-sheng Oct 12, 2021

Choose a reason for hiding this comment

kaiyan-sheng Oct 12, 2021

Choose a reason for hiding this comment

aspacca commented Oct 6, 2021 •

edited

Loading

elasticmachine commented Oct 6, 2021 •

edited by jenkins-beats-ci bot

Loading