Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement irsa server side support #7738

Merged
merged 47 commits into from
Jul 29, 2024
Merged

Conversation

nithyatsu
Copy link
Contributor

@nithyatsu nithyatsu commented Jul 15, 2024

Description

Add server side support for AWS IRSA.
UCP handles AWS resource deployment and needs irsa suuport.
Terraform provider communicates with AWS directly and needs IRSA support too.

Type of change

  • This pull request adds or changes features of Radius and has an approved issue (issue link required).

Partially Fixes: #7618

Copy link

codecov bot commented Jul 15, 2024

Codecov Report

Attention: Patch coverage is 50.00000% with 34 lines in your changes missing coverage. Please review.

Project coverage is 61.08%. Comparing base (93ddbdb) to head (007f7e6).

Files Patch % Lines
pkg/ucp/aws/ucpcredentialprovider.go 40.54% 22 Missing ⚠️
pkg/ucp/credentials/aws.go 0.00% 5 Missing ⚠️
pkg/ucp/credentials/azure.go 0.00% 4 Missing ⚠️
pkg/recipes/terraform/config/providers/aws.go 81.25% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7738      +/-   ##
==========================================
- Coverage   61.11%   61.08%   -0.03%     
==========================================
  Files         520      521       +1     
  Lines       27190    27229      +39     
==========================================
+ Hits        16618    16634      +16     
- Misses       9104     9131      +27     
+ Partials     1468     1464       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nithyatsu nithyatsu temporarily deployed to functional-tests July 15, 2024 19:52 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 15, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 662bd05
Unique ID func7706c522a6
Image tag pr-func7706c522a6
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func7706c522a6
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func7706c522a6
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func7706c522a6
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func7706c522a6
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting shared functional tests...
✅ msgrp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
❌ shared functional test failed. Please check the logs for more details

@nithyatsu nithyatsu changed the title Ucp irsa implement irsa server side support Jul 16, 2024
@nithyatsu nithyatsu force-pushed the ucp_irsa branch 4 times, most recently from 2fb90ae to 3db7d16 Compare July 19, 2024 18:05
@nithyatsu nithyatsu temporarily deployed to functional-tests July 19, 2024 18:48 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 19, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 63089ee
Unique ID func6eec8ca0e6
Image tag pr-func6eec8ca0e6
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func6eec8ca0e6
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func6eec8ca0e6
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func6eec8ca0e6
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func6eec8ca0e6
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting shared functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting ucp functional tests...
⌛ Starting datastoresrp functional tests...
✅ msgrp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ daprrp functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu temporarily deployed to functional-tests July 19, 2024 20:01 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 19, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 090c959
Unique ID funca907790858
Image tag pr-funca907790858
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funca907790858
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funca907790858
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funca907790858
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funca907790858
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting ucp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting daprrp functional tests...
✅ ucp functional tests succeeded
✅ msgrp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ samples functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu marked this pull request as ready for review July 19, 2024 20:10
@nithyatsu nithyatsu requested review from a team as code owners July 19, 2024 20:10
@nithyatsu nithyatsu temporarily deployed to functional-tests July 19, 2024 20:19 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 19, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 1e7cf7f
Unique ID funccdbb90194a
Image tag pr-funccdbb90194a
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funccdbb90194a
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funccdbb90194a
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funccdbb90194a
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funccdbb90194a
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting datastoresrp functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ ucp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
❌ shared functional test failed. Please check the logs for more details

@nithyatsu nithyatsu temporarily deployed to functional-tests July 19, 2024 21:28 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 19, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref cecfbb4
Unique ID funcb84800d4ce
Image tag pr-funcb84800d4ce
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcb84800d4ce
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcb84800d4ce
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcb84800d4ce
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcb84800d4ce
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting ucp functional tests...
⌛ Starting shared functional tests...
⌛ Starting cli functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting datastoresrp functional tests...
✅ kubernetes functional tests succeeded
✅ samples functional tests succeeded
✅ ucp functional tests succeeded
✅ msgrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded


// CredentialKind is IRSA
CredentialKindIRSA = "IRSA"
// CredentialKindAccessKey is AccessKey
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// CredentialKindAccessKey is AccessKey
// CredentialKind is AccessKey

})

t.Run("valid credential", func(t *testing.T) {
p := newMockProvider()
t.Run("valid redential", func(t *testing.T) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: credential

storage = c
default:
return nil, errors.New("invalid AWSAccessKeyCredentialProperties")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to combine both these?
eg. case *ucpapi.AwsAccessKeyCredentialProperties, *ucpapi.AwsIRSACredentialProperties

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice one. But it does not like it when I tried, since in the next line we have switch c := p.Storage.(type) and looks since now p can correspond to either of 2 values, it gives an error.

Copy link
Contributor

@lakshmimsft lakshmimsft Jul 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's pull common code into function and each switch case calls the function, eg:

func getStorageProperties(p any) (*InternalCredentialStorageProperties, error) {
	switch c := p.(type) {
	case *InternalCredentialStorageProperties:
		return c, nil
	default:
		return nil, errors.New("invalid AWS credential storage properties")
	}
}

pkg/recipes/terraform/config/providers/aws.go Outdated Show resolved Hide resolved
Comment on lines 164 to 172
// Radius requests will first be routed to STS endpoint,
// where it will be validated and then the request to the specific service (such as S3) will be made using
// the bearer token from the STS response.
// Based on the https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html,
// STS endpoint should be region based, and in the same region as
// Radius instance to minimize latency associated with STS call and thereby improve performance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems out of place because the code below isn't making any calls to AWS, was this intended?

Copy link
Contributor Author

@nithyatsu nithyatsu Jul 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added "once we switch to region based STS endpoint, we should add the region to the config." to the comment. Hope that makes it clearer. issues #7747 is at a high level going to introduce a way to set region for both ucp and terraform, based on radius's location.

pkg/recipes/terraform/config/providers/aws.go Outdated Show resolved Hide resolved
pkg/ucp/aws/ucpcredentialprovider.go Outdated Show resolved Hide resolved
nithyatsu added 18 commits July 29, 2024 14:02
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 29, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 7cded5a
Unique ID funcef1638b8e0
Image tag pr-funcef1638b8e0
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcef1638b8e0
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcef1638b8e0
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcef1638b8e0
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcef1638b8e0
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

Signed-off-by: nithyatsu <nithyasu@microsoft.com>
@nithyatsu nithyatsu temporarily deployed to functional-tests July 29, 2024 23:09 — with GitHub Actions Inactive
@kachawla kachawla temporarily deployed to functional-tests July 29, 2024 23:09 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 29, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref e6fe44f
Unique ID func61e4b1eff9
Image tag pr-func61e4b1eff9
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func61e4b1eff9
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func61e4b1eff9
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func61e4b1eff9
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func61e4b1eff9
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting corerp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
⌛ Starting datastoresrp-cloud functional tests...
✅ corerp-cloud functional tests succeeded
✅ ucp-cloud functional tests succeeded
✅ datastoresrp-cloud functional tests succeeded

@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 29, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository nithyatsu/radius
Commit ref 007f7e6
Unique ID func73023ff9ee
Image tag pr-func73023ff9ee
Click here to see the list of tools in the current test run
  • gotestsum 1.12.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func73023ff9ee
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func73023ff9ee
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func73023ff9ee
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func73023ff9ee
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp-cloud functional tests...
⌛ Starting ucp-cloud functional tests...
✅ datastoresrp-cloud functional tests succeeded
⌛ Starting corerp-cloud functional tests...
✅ ucp-cloud functional tests succeeded
✅ corerp-cloud functional tests succeeded

@kachawla kachawla merged commit c9b6581 into radius-project:main Jul 29, 2024
26 checks passed
Reshrahim pushed a commit to Reshrahim/radius that referenced this pull request Aug 27, 2024
# Description

Add server side support for AWS IRSA.
UCP handles AWS resource deployment and needs irsa suuport.
Terraform provider communicates with AWS directly and needs IRSA support
too.

## Type of change
- This pull request adds or changes features of Radius and has an
approved issue (issue link required).

Partially Fixes: radius-project#7618

---------

Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: Nithya Subramanian <98416062+nithyatsu@users.noreply.github.com>
Co-authored-by: Karishma Chawla <kachawla@microsoft.com>
Signed-off-by: Reshma Abdul Rahim <reshmarahim.abdul@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add IRSA (workload identity) support for AWS cloud provider
4 participants