feat(stepfunctions): execution history logging options #6933

workeitel · 2020-03-22T21:06:46Z

Commit Message

feat(stepfunctions): execution history logging options (#6933)

closes #5754

End Commit Message

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation · 2020-03-22T21:54:37Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 0f356da
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-22T22:01:38Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 5842aab
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

nija-at · 2020-03-25T13:41:18Z

packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

+            // https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html#cloudwatch-iam-policy
+            this.addToRolePolicy(new iam.PolicyStatement({
+              effect: iam.Effect.ALLOW,
+              actions: [
+                "logs:CreateLogDelivery",
+                "logs:GetLogDelivery",
+                "logs:UpdateLogDelivery",
+                "logs:DeleteLogDelivery",
+                "logs:ListLogDeliveries",
+                "logs:PutResourcePolicy",
+                "logs:DescribeResourcePolicies",
+                "logs:DescribeLogGroups"
+              ],
+              resources: ["*"]
+            }));


We try our best to keep least privilege, so avoid * resource.

Use loggroup.grant() API instead.

Those permissions are not bound to resources but more general CloudWatch Logs permissions allowing StepFunctions to write into the account: https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazoncloudwatchlogs.html

There is no resource which can be referenced.

Weirdly, I can't seem to find any documentation on log delivery, so I can't quite say what this is or what it's about. I'll try to gather more information.

On the other hand, the ability to PutResourcePolicy across all log groups in the account seems a bit excessive. Could you check if the following works instead -

In addition to the policy around log delivery

effect: iam.Effect.ALLOW, actions: [ 'PutLogEvents', 'GetLogEvents' ], resources: [ `${conf.destination.logGroupArn}:log-stream:*` ]

As I mentioned the permissions are not bound to a resource but for the whole account. That's also shown in the docs I linked. It does not work with a LogGroup/LogStream restriction.

packages/@aws-cdk/aws-stepfunctions/test/integ.stepfunctions-logging.ts

packages/@aws-cdk/aws-stepfunctions/test/test.state-machine.ts

nija-at · 2020-03-25T13:44:30Z

Thanks for your contribution!

Once you have addressed these changes and the PR is ready for review, dismiss my review and it'll show up in my inbox.

aws-cdk-automation · 2020-03-25T14:16:22Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: c1660bf
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T14:18:45Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 91bb310
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T14:18:51Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 8fb79a3
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T14:19:03Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: e717d2b
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T14:28:51Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 5f09945
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T14:57:24Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 3346b4e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T15:39:16Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 811a9cb
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-03-25T16:40:25Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: cc7210e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

nija-at · 2020-03-27T10:34:56Z

packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

+            // https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html#cloudwatch-iam-policy
+            this.addToRolePolicy(new iam.PolicyStatement({
+              effect: iam.Effect.ALLOW,
+              actions: [
+                "logs:CreateLogDelivery",
+                "logs:GetLogDelivery",
+                "logs:UpdateLogDelivery",
+                "logs:DeleteLogDelivery",
+                "logs:ListLogDeliveries",
+                "logs:PutResourcePolicy",
+                "logs:DescribeResourcePolicies",
+                "logs:DescribeLogGroups"
+              ],
+              resources: ["*"]
+            }));


Weirdly, I can't seem to find any documentation on log delivery, so I can't quite say what this is or what it's about. I'll try to gather more information.

On the other hand, the ability to PutResourcePolicy across all log groups in the account seems a bit excessive. Could you check if the following works instead -

In addition to the policy around log delivery

effect: iam.Effect.ALLOW, actions: [ 'PutLogEvents', 'GetLogEvents' ], resources: [ `${conf.destination.logGroupArn}:log-stream:*` ]

packages/@aws-cdk/aws-stepfunctions/README.md

packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

Fixes aws#5754

Co-Authored-By: Niranjan Jayakar <nija@amazon.com>

Not needed according to the PR feedback.

Pull request has been modified.

aws-cdk-automation · 2020-03-30T22:26:19Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: bb6f9fd
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2020-04-01T08:00:40Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

aws-cdk-automation · 2020-04-01T08:51:26Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: d886361
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

aws-cdk-automation · 2020-04-01T09:44:49Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 47646ba
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Pull request has been modified.

aws-cdk-automation · 2020-04-02T20:05:13Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 756e62d
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

nija-at · 2020-04-03T08:56:47Z

@workeitel - I'm going to approve this PR again. Please don't click on 'Update branch'. The bot will take care of everything.

mergify · 2020-04-03T08:57:32Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

aws-cdk-automation · 2020-04-03T09:28:35Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
Commit ID: 6eec49b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2020-04-03T09:28:54Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

workeitel force-pushed the stepfunctions-logging branch from 0f356da to 5842aab Compare March 22, 2020 21:16

SomayaB assigned nija-at Mar 25, 2020

nija-at previously requested changes Mar 25, 2020

View reviewed changes

workeitel requested a review from nija-at March 25, 2020 16:40

aws deleted a comment from mergify bot Mar 26, 2020

nija-at previously requested changes Mar 30, 2020

View reviewed changes

workeitel and others added 9 commits March 30, 2020 23:31

feat(stepfunctions): Support logging configuration

d472177

Fixes aws#5754

Update packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

2366d9d

Co-Authored-By: Niranjan Jayakar <nija@amazon.com>

Update packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

3e31990

Co-Authored-By: Niranjan Jayakar <nija@amazon.com>

Update packages/@aws-cdk/aws-stepfunctions/lib/state-machine.ts

2c5962e

Co-Authored-By: Niranjan Jayakar <nija@amazon.com>

Update packages/@aws-cdk/aws-stepfunctions/test/test.state-machine.ts

c3cfcd3

Co-Authored-By: Niranjan Jayakar <nija@amazon.com>

Remove integration test for step functions logging

12e23aa

Not needed according to the PR feedback.

Incorporate PR feedback

b8bf577

Fix README.md

003b63d

PR fixes

bb6f9fd

workeitel force-pushed the stepfunctions-logging branch from cc7210e to bb6f9fd Compare March 30, 2020 21:40

nija-at previously approved these changes Apr 1, 2020

View reviewed changes

Merge branch 'master' into stepfunctions-logging

d886361

nija-at changed the title ~~feat(stepfunctions): Support logging configuration~~ feat(stepfunctions): execution history logging configuration Apr 1, 2020

nija-at changed the title ~~feat(stepfunctions): execution history logging configuration~~ feat(stepfunctions): execution history logging options Apr 1, 2020

Merge branch 'master' into stepfunctions-logging

47646ba

Merge branch 'master' into stepfunctions-logging

756e62d

workeitel requested a review from nija-at April 2, 2020 20:15

nija-at approved these changes Apr 3, 2020

View reviewed changes

Merge branch 'master' into stepfunctions-logging

6eec49b

mergify bot merged commit adb6954 into aws:master Apr 3, 2020

ccfife mentioned this pull request Apr 21, 2020

📊Tracking: Step Functions #6489

Closed

24 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(stepfunctions): execution history logging options #6933

feat(stepfunctions): execution history logging options #6933

workeitel commented Mar 22, 2020 •

edited by nija-at

Loading

aws-cdk-automation commented Mar 22, 2020

aws-cdk-automation commented Mar 22, 2020

nija-at Mar 25, 2020

workeitel Mar 25, 2020

nija-at Mar 27, 2020

workeitel Mar 30, 2020

nija-at commented Mar 25, 2020 •

edited

Loading

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

aws-cdk-automation commented Mar 25, 2020

nija-at Mar 27, 2020

aws-cdk-automation commented Mar 30, 2020

mergify bot commented Apr 1, 2020

aws-cdk-automation commented Apr 1, 2020

aws-cdk-automation commented Apr 1, 2020

aws-cdk-automation commented Apr 2, 2020

nija-at commented Apr 3, 2020

mergify bot commented Apr 3, 2020

aws-cdk-automation commented Apr 3, 2020

mergify bot commented Apr 3, 2020

feat(stepfunctions): execution history logging options #6933

feat(stepfunctions): execution history logging options #6933

Conversation

workeitel commented Mar 22, 2020 • edited by nija-at Loading

Commit Message

End Commit Message

aws-cdk-automation commented Mar 22, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 22, 2020

AWS CodeBuild CI Report

nija-at Mar 25, 2020

Choose a reason for hiding this comment

workeitel Mar 25, 2020

Choose a reason for hiding this comment

nija-at Mar 27, 2020

Choose a reason for hiding this comment

workeitel Mar 30, 2020

Choose a reason for hiding this comment

nija-at commented Mar 25, 2020 • edited Loading

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Mar 25, 2020

AWS CodeBuild CI Report

nija-at Mar 27, 2020

Choose a reason for hiding this comment

aws-cdk-automation commented Mar 30, 2020

AWS CodeBuild CI Report

mergify bot commented Apr 1, 2020

aws-cdk-automation commented Apr 1, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Apr 1, 2020

AWS CodeBuild CI Report

aws-cdk-automation commented Apr 2, 2020

AWS CodeBuild CI Report

nija-at commented Apr 3, 2020

mergify bot commented Apr 3, 2020

aws-cdk-automation commented Apr 3, 2020

AWS CodeBuild CI Report

mergify bot commented Apr 3, 2020

workeitel commented Mar 22, 2020 •

edited by nija-at

Loading

nija-at commented Mar 25, 2020 •

edited

Loading