Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement controllers for AWS IRSA #7735

Closed
wants to merge 19 commits into from
Closed

Conversation

nithyatsu
Copy link
Contributor

@nithyatsu nithyatsu commented Jul 10, 2024

Description

Add convertor and controllers for the new IRSA credential type

Type of change

  • This pull request adds or changes features of Radius and has an approved issue (issue link required).

Partially Fixes: #7618

nithyatsu added 11 commits July 10, 2024 10:07
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
@nithyatsu nithyatsu force-pushed the irsa_model_controller2 branch from cd8eb33 to 32053b1 Compare July 10, 2024 17:07
@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 17:07 — with GitHub Actions Inactive
Signed-off-by: nithyatsu <nithyasu@microsoft.com>
@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 17:18 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref 32053b1
Unique ID funca1b70a90bc
Image tag pr-funca1b70a90bc
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funca1b70a90bc
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funca1b70a90bc
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funca1b70a90bc
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funca1b70a90bc
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting samples functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting ucp functional tests...
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ ucp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref e87b0eb
Unique ID func572b4aab1a
Image tag pr-func572b4aab1a
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func572b4aab1a
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func572b4aab1a
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func572b4aab1a
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func572b4aab1a
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting cli functional tests...
⌛ Starting ucp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting msgrp functional tests...
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ msgrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

Copy link

github-actions bot commented Jul 10, 2024

Unit Tests

3 264 tests  +2   3 258 ✅ +2   4m 1s ⏱️ -8s
  261 suites ±0       6 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit bbd1bca. ± Comparison against base commit 0782511.

This pull request removes 2 and adds 4 tests. Note that renamed tests count towards both.
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertDataModelToVersioned/credentialresourcedatamodel-aws.json
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertVersionedToDataModel/credentialresource-aws.json
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertDataModelToVersioned/credentialresourcedatamodel-aws-accesskey.json
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertDataModelToVersioned/credentialresourcedatamodel-aws-irsa.json
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertVersionedToDataModel/credentialresource-aws-accesskey.json
github.com/radius-project/radius/pkg/ucp/api/v20231001preview ‑ TestAWSCredentialConvertVersionedToDataModel/credentialresource-aws-irsa.json

♻️ This comment has been updated with latest results.

Copy link

codecov bot commented Jul 10, 2024

Codecov Report

Attention: Patch coverage is 73.33333% with 16 lines in your changes missing coverage. Please review.

Project coverage is 61.26%. Comparing base (ccc385f) to head (bbd1bca).
Report is 3 commits behind head on main.

Files Patch % Lines
pkg/recipes/terraform/config/providers/aws.go 40.00% 7 Missing and 2 partials ⚠️
.../api/v20231001preview/aws_credential_conversion.go 84.21% 5 Missing and 1 partial ⚠️
...ler/credentials/aws/createorupdateawscredential.go 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7735      +/-   ##
==========================================
+ Coverage   61.00%   61.26%   +0.25%     
==========================================
  Files         520      520              
  Lines       27010    27441     +431     
==========================================
+ Hits        16478    16811     +333     
- Misses       9080     9166      +86     
- Partials     1452     1464      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@nithyatsu nithyatsu marked this pull request as ready for review July 10, 2024 17:52
@nithyatsu nithyatsu requested review from a team as code owners July 10, 2024 17:52
Signed-off-by: Nithya Subramanian <98416062+nithyatsu@users.noreply.github.com>
@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 17:54 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref 2db30d1
Unique ID funcc6db39c2c5
Image tag pr-funcc6db39c2c5
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-funcc6db39c2c5
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-funcc6db39c2c5
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-funcc6db39c2c5
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-funcc6db39c2c5
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting cli functional tests...
⌛ Starting ucp functional tests...
⌛ Starting shared functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting datastoresrp functional tests...
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ cli functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 19:58 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref 5446761
Unique ID func61ac60c7d2
Image tag pr-func61ac60c7d2
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func61ac60c7d2
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func61ac60c7d2
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func61ac60c7d2
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func61ac60c7d2
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting ucp functional tests...
⌛ Starting cli functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting samples functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting datastoresrp functional tests...
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 21:10 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref 2aee7f2
Unique ID func4b5166225c
Image tag pr-func4b5166225c
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func4b5166225c
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func4b5166225c
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func4b5166225c
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func4b5166225c
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting cli functional tests...
✅ samples functional tests succeeded
✅ msgrp functional tests succeeded
✅ ucp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ daprrp functional tests succeeded
❌ datastoresrp functional test cancelled. Please check the logs for more details
❌ shared functional test cancelled. Please check the logs for more details
❌ cli functional test cancelled. Please check the logs for more details
⌛ Starting cli functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting shared functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting datastoresrp functional tests...
✅ kubernetes functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 22:48 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref c3910f5
Unique ID func39294eea79
Image tag pr-func39294eea79
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func39294eea79
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func39294eea79
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func39294eea79
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func39294eea79
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting cli functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting ucp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting shared functional tests...
✅ msgrp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

@nithyatsu nithyatsu temporarily deployed to functional-tests July 10, 2024 23:04 — with GitHub Actions Inactive
@radius-functional-tests
Copy link

radius-functional-tests bot commented Jul 10, 2024

Radius functional test overview

🔍 Go to test action run

Name Value
Repository radius-project/radius
Commit ref bbd1bca
Unique ID func06e95ceb93
Image tag pr-func06e95ceb93
Click here to see the list of tools in the current test run
  • gotestsum 1.10.0
  • KinD: v0.20.0
  • Dapr: 1.12.0
  • Azure KeyVault CSI driver: 1.4.2
  • Azure Workload identity webhook: 1.1.0
  • Bicep recipe location ghcr.io/radius-project/dev/test/testrecipes/test-bicep-recipes/<name>:pr-func06e95ceb93
  • Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
  • applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-func06e95ceb93
  • controller test image location: ghcr.io/radius-project/dev/controller:pr-func06e95ceb93
  • ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-func06e95ceb93
  • deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting cli functional tests...
⌛ Starting ucp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting samples functional tests...
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ daprrp functional tests succeeded
✅ cli functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

Copy link
Contributor

@vishwahiremat vishwahiremat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nithyatsu nithyatsu closed this Jul 15, 2024
@nithyatsu nithyatsu reopened this Jul 15, 2024
@nithyatsu nithyatsu closed this Jul 15, 2024
@nithyatsu nithyatsu reopened this Jul 15, 2024
@nithyatsu nithyatsu closed this Jul 15, 2024
pkg/recipes/terraform/config/providers/aws.go Show resolved Hide resolved
}
case ucp_datamodel.AWSIRSACredentialKind:
if credentials.IRSACredential == nil || credentials.IRSACredential.RoleARN == "" {
logger.Info("AWS IRSACredential is not registered, skipping credentials configuration.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should IRSACredential be one word or can it be split?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ytimocin I am not sure what is meant by split. Could you please elaborate a bit.

Comment on lines +39 to 43
AccessKeyCredential: &ucp_datamodel.AWSAccessKeyCredentialProperties{
AccessKeyID: "testAccessKey",
SecretAccessKey: "testSecretKey",
},
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have IRSACredential unit tests?

Copy link
Contributor Author

@nithyatsu nithyatsu Jul 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I going to add them as part of Terraform RP changes. I had to add minimal changes in the PR so that it would compile with new data model. But I have not yet implemented Terraform support.

},
}
case nil:
return nil, &v1.ErrModelConversion{PropertyName: "$.properties.storage", ValidValue: "not nil"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of not nil should we have a proper error message?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have this since storage is a structure like below:
"storage":{
"kind":"Internal"
}

We seem to return the "not nil" error message for all structures, for example

case *InternalCredentialStorageProperties:
			if c.Kind == nil {
				return nil, &v1.ErrModelConversion{PropertyName: "$.properties", ValidValue: "not nil"}
			}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you reformat the JSON files?

Comment on lines +47 to +51
Kind: ucp_datamodel.AWSAccessKeyCredentialKind,
AccessKeyCredential: &ucp_datamodel.AWSAccessKeyCredentialProperties{
AccessKeyID: "fakeid",
SecretAccessKey: "fakesecretkey",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have IRSA unit tests here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IRSA UT are coming as part of UCP PR for supporting IRSA. (similar to terraform)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we delete these files? Are we going to replace them?

Copy link
Contributor Author

@nithyatsu nithyatsu Jul 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, earlier we had just one kind so this file actually contained the accesskey credential example. I deleted it ( renamed to accesskey) and added the irsa example.json, as part of tsp PR (https://github.com/radius-project/radius/pull/7708/files) . I had missed deleting these files though.

@nithyatsu nithyatsu reopened this Jul 15, 2024
@nithyatsu nithyatsu closed this Jul 15, 2024
@nithyatsu
Copy link
Contributor Author

continuing the PR in #7739

@nithyatsu nithyatsu reopened this Jul 15, 2024
@nithyatsu nithyatsu closed this Jul 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add IRSA (workload identity) support for AWS cloud provider
3 participants