-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(stepfunctions-tasks): incorrect S3 permissions for AthenaStartQueryExecution #11203
Conversation
…rd S3 permissions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sumeet-Badyal thanks for submitting the fix - I updated the commit message to indicate the error as per our CONTRIBUTING
guide.
the rationale for that is because this will show up in the CHANGELOG
under bug fixes and it reads better when it describes the problem that was fixed.
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Outdated
Show resolved
Hide resolved
packages/@aws-cdk/aws-stepfunctions-tasks/test/athena/start-query-execution.test.ts
Outdated
Show resolved
Hide resolved
Pull request has been modified.
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Outdated
Show resolved
Hide resolved
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Show resolved
Hide resolved
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Outdated
Show resolved
Hide resolved
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sumeet-Badyal I updated the commit description to include the breaking change entry so that it shows up in the changelog.
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Outdated
Show resolved
Hide resolved
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Show resolved
Hide resolved
Pull request has been modified.
packages/@aws-cdk/aws-stepfunctions-tasks/lib/athena/start-query-execution.ts
Show resolved
Hide resolved
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
AWS CodeBuild CI Report
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository |
Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork). |
's3:ListBucketMultipartUploads', | ||
's3:ListMultipartUploadParts', | ||
's3:PutObject'], | ||
resources: [this.props.resultConfiguration?.outputLocation ?? '*'], // Need S3 location where data is stored https://docs.aws.amazon.com/athena/latest/ug/security-iam-athena.html | ||
resources: [this.props.resultConfiguration?.outputLocation?.bucketName ? `arn:aws:s3:::${this.props.resultConfiguration?.outputLocation?.bucketName}/${this.props.resultConfiguration?.outputLocation?.objectKey}/*` : '*'], // Need S3 location where data is stored or Athena throws an Unable to verify/create output bucket https://docs.aws.amazon.com/athena/latest/ug/security-iam-athena.html |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Sumeet-Badyal Thank you for your PR!
I've tried the changes in this PR, but since objectKey
is required, it doesn't seem to work when specifying into directly under the bucket.
Another problem might be that when using CREATE TABLE AS, files are created under tables/
directory of the bucket specified with outputLocation
, so we have to specify arn:aws:s3:::my-bucket/*
(not including any object keys except wildcard) instead of arn:aws:s3:::my-bucket/foo/*
.
The changes made by #11045 grant S3 scoped permissions if the optional parameter output location is passed. The output location is not parsed correctly to be attached as a resource. When the output location is correctly parsed, state machines with a valid definition and a valid S3 bucket still fail due to an
Unable to verify/create output bucket
error. The exact same state machine and S3 bucket pass with wildcard permissions as such the resource for Start Query Execution must be changed to*
.BREAKING CHANGE: type of
outputLocation
in the experimental AthenaStartQueryExecution
has been changed tos3.Location
fromstring
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license