Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing support for Web Identity Tokens #475

Closed
otterley opened this issue Jan 19, 2020 · 5 comments · Fixed by #488 or #507
Closed

Missing support for Web Identity Tokens #475

otterley opened this issue Jan 19, 2020 · 5 comments · Fixed by #488 or #507

Comments

@otterley
Copy link

v2 of the Go SDK is missing support for Web Identity Tokens. (It's present in SDK v1.)

Support for Web Identity Tokens is particularly crucial for Go services running in Amazon EKS, where IAM Roles for Service Accounts are now the officially-supported way for Kubernetes Pods to obtain AWS credentials without having to grant the entire EC2 instance (i.e., node) access via an EC2 Instance Role. This enables least-privilege access in multi-tenant environments. See https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html for details.

See aws/aws-sdk-go@2e1d76a for the commit that added support to v1 of the Go SDK.

@hasheddan
Copy link
Contributor

@jasdel do y'all have a timeline on when you would like this implemented? I would be happy to do the work 👍

@hasheddan
Copy link
Contributor

@otterley @owlwalks I was able to develop a work around here that can be used until this feature is implemented. Here is an example for use with an RDS client:

func NewClient() (Client, error) {
	cfg, err := external.LoadDefaultAWSConfig()
	if err != nil {
		return nil, err
	}
	svc := sts.New(cfg)

	b, err := ioutil.ReadFile(os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE"))
	if err != nil {
		return nil, err
	}
	token := string(b)
	sess := strconv.FormatInt(time.Now().UnixNano(), 10)
	role := os.Getenv("AWS_ROLE_ARN")
	resp, err := svc.AssumeRoleWithWebIdentityRequest(
		&sts.AssumeRoleWithWebIdentityInput{
			RoleSessionName:  &sess,
			WebIdentityToken: &token,
			RoleArn:          &role,
		},
	).Send(context.TODO())
	if err != nil {
		return nil, err
	}
	creds := aws.Credentials{
		AccessKeyID:     aws.StringValue(resp.Credentials.AccessKeyId),
		SecretAccessKey: aws.StringValue(resp.Credentials.SecretAccessKey),
		SessionToken:    aws.StringValue(resp.Credentials.SessionToken),
	}
	shared := external.SharedConfig{
		Credentials: creds,
		Region:      os.Getenv("AWS_REGION"),
	}
	config, err := external.LoadDefaultAWSConfig(shared)
	if err != nil {
		return nil, err
	}
	return rds.New(config), err
}

Happy to answer any questions!

@skmcgrail
Copy link
Member

The work to port this credential provider and a few other improvements should be landing soon. This is currently planned to be implemented as part of our sprint. Will update this once that PR is out for review.

@hasheddan
Copy link
Contributor

@skmcgrail thanks for following up here! Let me know if I can be of any help

@skmcgrail skmcgrail linked a pull request Feb 7, 2020 that will close this issue
6 tasks
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this issue Mar 17, 2020
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [aws#249](aws#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [aws#475](aws#475)
    * Fixes [aws#338](aws#338)
  * Adds Support for `credential_source`
    * Fixes [aws#274](aws#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487))
* `aws`: Updates SDK retry behavior ([aws#487](aws#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503)
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
skmcgrail added a commit to skmcgrail/aws-sdk-go-v2 that referenced this issue Mar 17, 2020
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([aws#488](aws#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [aws#249](aws#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [aws#475](aws#475)
    * Fixes [aws#338](aws#338)
  * Adds Support for `credential_source`
    * Fixes [aws#274](aws#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([aws#487](aws#487))
* `aws`: Updates SDK retry behavior ([aws#487](aws#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([aws#503](aws#503))
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
skmcgrail added a commit that referenced this issue Mar 17, 2020
Breaking Change
---
* Update SDK retry behavior
  * Significant updates were made the SDK's retry behavior. The SDK will now retry all connections error. In addition, to changing what errors are retried the SDK's retry behavior not distinguish the difference between throttling errors, and regular retryable errors. All errors will be retried with the same backoff jitter delay scaling.
  * The SDK will attempt an operation request 3 times by default. This is one less than the previous initial request with 3 retires.
  * New helper functions in the new `aws/retry` package allow wrapping a `Retrier` with custom behavior, overriding the base `Retrier`, (e.g. `AddWithErrorCodes`, and `AddWithMaxAttempts`)
* Update SDK error handling
  * Updates the SDK's handling of errors to take advantage of Go 1.13's new `errors.As`, `Is`, and `Unwrap`. The SDK's errors were updated to satisfy the `Unwrap` interface, returning the underlying error.
  * With this update, you can now more easily access the SDK's layered errors, and meaningful state such as, `Timeout`, `Temporary`, and other states added to the SDK such as `CanceledError`.
* Bump SDK minimum supported version from Go 1.12 to Go 1.13
  * The SDK's minimum supported version is bumped to take advantage of Go 1.13's updated `errors` package.

Services
---
* Synced the V2 SDK with latest AWS service API definitions.

SDK Features
---
* `aws`: Add Support for additional credential providers and credential configuration chaining ([#488](#488))
  * `aws/processcreds`: Adds Support for the Process Credential Provider
    * Fixes [#249](#249)
  * `aws/stscreds`: Adds Support for the Web Identity Credential Provider
    * Fixes [#475](#475)
    * Fixes [#338](#338)
  * Adds Support for `credential_source`
    * Fixes [#274](#274)
* `aws/awserr`: Adds support for Go 1.13's `errors.Unwrap` ([#487](#487))
* `aws`: Updates SDK retry behavior ([#487](#487))
  * `aws/retry`: New package defining logic to determine if a request should be retried, and backoff.
  * `aws/ratelimit`: New package defining rate limit logic such as token bucket used by the `retry.Standard` retrier.

SDK Enhancements
---
* `aws`: Add grouping of concurrent refresh of credentials ([#503](#503))
  * Concurrent calls to `Retrieve` are now grouped in order to prevent numerous synchronous calls to refresh the credentials. Replacing the mutex with a singleflight reduces the overall amount of time request signatures need to wait while retrieving credentials. This is improvement becomes pronounced when many requests are made concurrently.
* `service/s3/s3manager`: Improve memory allocation behavior by replacing sync.Pool with custom pool implementation
  * Improves memory allocations that occur when the provided `io.Reader` to upload does not satisfy both the `io.ReaderAt` and `io.ReadSeeker` interfaces.

SDK Bugs
---
* `service/s3/s3manager`: Fix resource leaks when the following occurred:
  * Failed CreateMultipartUpload call
  * Failed UploadPart call
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants