-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(stepfunctions-tasks): start a nested state machine execution as a construct #8178
Merged
Merged
Changes from all commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
a4573fe
feat(stepfunctions-tasks): start a Step Functions execution as a task
shivlaks e45c51a
add new class, unit tests, and integ tests
shivlaks e2d022b
drop todo that has been addressed
shivlaks 28ce0a5
fix up readme
shivlaks f293e7e
mark older implementation of StartExecution as deprecated
shivlaks 83eb1d2
cleanup
shivlaks 12a3189
Merge branch 'master' into shivlaks/sfn-tasks-sfn-integration
shivlaks aee6c47
change the default input to nested state machine to be the state inpu…
shivlaks 0601585
Merge branch 'master' into shivlaks/sfn-tasks-sfn-integration
shivlaks 678bd34
feedback from PR
shivlaks 33c6520
Merge branch 'master' into shivlaks/sfn-tasks-sfn-integration
shivlaks a9fbe07
Merge branch 'master' into shivlaks/sfn-tasks-sfn-integration
mergify[bot] File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
129 changes: 129 additions & 0 deletions
129
packages/@aws-cdk/aws-stepfunctions-tasks/lib/stepfunctions/start-execution.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,129 @@ | ||
import * as iam from '@aws-cdk/aws-iam'; | ||
import * as sfn from '@aws-cdk/aws-stepfunctions'; | ||
import { Construct, Stack } from '@aws-cdk/core'; | ||
import { integrationResourceArn, validatePatternSupported } from '../private/task-utils'; | ||
|
||
/** | ||
* Properties for StartExecution | ||
*/ | ||
export interface StepFunctionsStartExecutionProps extends sfn.TaskStateBaseProps { | ||
/** | ||
* The Step Functions state machine to start the execution on. | ||
*/ | ||
readonly stateMachine: sfn.IStateMachine; | ||
|
||
/** | ||
* The JSON input for the execution, same as that of StartExecution. | ||
* | ||
* @see https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartExecution.html | ||
* | ||
* @default - The state input (JSON path '$') | ||
*/ | ||
readonly input?: sfn.TaskInput; | ||
|
||
/** | ||
* The name of the execution, same as that of StartExecution. | ||
* | ||
* @see https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartExecution.html | ||
* | ||
* @default - None | ||
*/ | ||
readonly name?: string; | ||
} | ||
|
||
/** | ||
* A Step Functions Task to call StartExecution on another state machine. | ||
* | ||
* It supports three service integration patterns: FIRE_AND_FORGET, SYNC and WAIT_FOR_TASK_TOKEN. | ||
*/ | ||
export class StepFunctionsStartExecution extends sfn.TaskStateBase { | ||
private static readonly SUPPORTED_INTEGRATION_PATTERNS = [ | ||
sfn.IntegrationPattern.REQUEST_RESPONSE, | ||
sfn.IntegrationPattern.RUN_JOB, | ||
sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN, | ||
]; | ||
|
||
protected readonly taskMetrics?: sfn.TaskMetricsConfig; | ||
protected readonly taskPolicies?: iam.PolicyStatement[]; | ||
|
||
private readonly integrationPattern: sfn.IntegrationPattern; | ||
|
||
constructor(scope: Construct, id: string, private readonly props: StepFunctionsStartExecutionProps) { | ||
super(scope, id, props); | ||
|
||
this.integrationPattern = props.integrationPattern || sfn.IntegrationPattern.REQUEST_RESPONSE; | ||
validatePatternSupported(this.integrationPattern, StepFunctionsStartExecution.SUPPORTED_INTEGRATION_PATTERNS); | ||
|
||
if (this.integrationPattern === sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN && !sfn.FieldUtils.containsTaskToken(props.input)) { | ||
throw new Error('Task Token is required in `input` for callback. Use Context.taskToken to set the token.'); | ||
} | ||
|
||
this.taskPolicies = this.createScopedAccessPolicy(); | ||
} | ||
|
||
protected renderTask(): any { | ||
// suffix of ':2' indicates that the output of the nested state machine should be JSON | ||
// suffix is only applicable when waiting for a nested state machine to complete (RUN_JOB) | ||
// https://docs.aws.amazon.com/step-functions/latest/dg/connect-stepfunctions.html | ||
const suffix = this.integrationPattern === sfn.IntegrationPattern.RUN_JOB ? ':2' : ''; | ||
return { | ||
Resource: `${integrationResourceArn('states', 'startExecution', this.integrationPattern)}${suffix}`, | ||
Parameters: sfn.FieldUtils.renderObject({ | ||
Input: this.props.input ? this.props.input.value : sfn.TaskInput.fromDataAt('$').value, | ||
StateMachineArn: this.props.stateMachine.stateMachineArn, | ||
Name: this.props.name, | ||
}), | ||
}; | ||
} | ||
|
||
/** | ||
* As StateMachineArn is extracted automatically from the state machine object included in the constructor, | ||
* | ||
* the scoped access policy should be generated accordingly. | ||
* | ||
* This means the action of StartExecution should be restricted on the given state machine, instead of being granted to all the resources (*). | ||
*/ | ||
private createScopedAccessPolicy(): iam.PolicyStatement[] { | ||
const stack = Stack.of(this); | ||
|
||
const policyStatements = [ | ||
new iam.PolicyStatement({ | ||
actions: ['states:StartExecution'], | ||
resources: [this.props.stateMachine.stateMachineArn], | ||
}), | ||
]; | ||
|
||
// Step Functions use Cloud Watch managed rules to deal with synchronous tasks. | ||
if (this.integrationPattern === sfn.IntegrationPattern.RUN_JOB) { | ||
policyStatements.push( | ||
new iam.PolicyStatement({ | ||
actions: ['states:DescribeExecution', 'states:StopExecution'], | ||
// https://docs.aws.amazon.com/step-functions/latest/dg/concept-create-iam-advanced.html#concept-create-iam-advanced-execution | ||
resources: [ | ||
stack.formatArn({ | ||
service: 'states', | ||
resource: 'execution', | ||
sep: ':', | ||
resourceName: `${stack.parseArn(this.props.stateMachine.stateMachineArn, ':').resourceName}*`, | ||
}), | ||
], | ||
}), | ||
); | ||
|
||
policyStatements.push( | ||
new iam.PolicyStatement({ | ||
actions: ['events:PutTargets', 'events:PutRule', 'events:DescribeRule'], | ||
resources: [ | ||
stack.formatArn({ | ||
service: 'events', | ||
resource: 'rule', | ||
resourceName: 'StepFunctionsGetEventsForStepFunctionsExecutionRule', | ||
}), | ||
], | ||
}), | ||
); | ||
} | ||
|
||
return policyStatements; | ||
} | ||
} |
187 changes: 187 additions & 0 deletions
187
...s/@aws-cdk/aws-stepfunctions-tasks/test/stepfunctions/integ.start-execution.expected.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,187 @@ | ||
{ | ||
"Resources": { | ||
"ChildRole1E3E0EF5": { | ||
"Type": "AWS::IAM::Role", | ||
"Properties": { | ||
"AssumeRolePolicyDocument": { | ||
"Statement": [ | ||
{ | ||
"Action": "sts:AssumeRole", | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": { | ||
"Fn::Join": [ | ||
"", | ||
[ | ||
"states.", | ||
{ | ||
"Ref": "AWS::Region" | ||
}, | ||
".amazonaws.com" | ||
] | ||
] | ||
} | ||
} | ||
} | ||
], | ||
"Version": "2012-10-17" | ||
} | ||
} | ||
}, | ||
"ChildDAB30558": { | ||
"Type": "AWS::StepFunctions::StateMachine", | ||
"Properties": { | ||
"DefinitionString": "{\"StartAt\":\"Pass\",\"States\":{\"Pass\":{\"Type\":\"Pass\",\"End\":true}}}", | ||
"RoleArn": { | ||
"Fn::GetAtt": ["ChildRole1E3E0EF5", "Arn"] | ||
} | ||
}, | ||
"DependsOn": ["ChildRole1E3E0EF5"] | ||
}, | ||
"ParentRole5F0C366C": { | ||
"Type": "AWS::IAM::Role", | ||
"Properties": { | ||
"AssumeRolePolicyDocument": { | ||
"Statement": [ | ||
{ | ||
"Action": "sts:AssumeRole", | ||
"Effect": "Allow", | ||
"Principal": { | ||
"Service": { | ||
"Fn::Join": [ | ||
"", | ||
[ | ||
"states.", | ||
{ | ||
"Ref": "AWS::Region" | ||
}, | ||
".amazonaws.com" | ||
] | ||
] | ||
} | ||
} | ||
} | ||
], | ||
"Version": "2012-10-17" | ||
} | ||
} | ||
}, | ||
"ParentRoleDefaultPolicy9BDC56DC": { | ||
"Type": "AWS::IAM::Policy", | ||
"Properties": { | ||
"PolicyDocument": { | ||
"Statement": [ | ||
{ | ||
"Action": "states:StartExecution", | ||
"Effect": "Allow", | ||
"Resource": { | ||
"Ref": "ChildDAB30558" | ||
} | ||
}, | ||
{ | ||
"Action": ["states:DescribeExecution", "states:StopExecution"], | ||
"Effect": "Allow", | ||
"Resource": { | ||
"Fn::Join": [ | ||
"", | ||
[ | ||
"arn:", | ||
{ | ||
"Ref": "AWS::Partition" | ||
}, | ||
":states:", | ||
{ | ||
"Ref": "AWS::Region" | ||
}, | ||
":", | ||
{ | ||
"Ref": "AWS::AccountId" | ||
}, | ||
":execution:", | ||
{ | ||
"Fn::Select": [ | ||
6, | ||
{ | ||
"Fn::Split": [ | ||
":", | ||
{ | ||
"Ref": "ChildDAB30558" | ||
} | ||
] | ||
} | ||
] | ||
}, | ||
"*" | ||
] | ||
] | ||
} | ||
}, | ||
{ | ||
"Action": ["events:PutTargets", "events:PutRule", "events:DescribeRule"], | ||
"Effect": "Allow", | ||
"Resource": { | ||
"Fn::Join": [ | ||
"", | ||
[ | ||
"arn:", | ||
{ | ||
"Ref": "AWS::Partition" | ||
}, | ||
":events:", | ||
{ | ||
"Ref": "AWS::Region" | ||
}, | ||
":", | ||
{ | ||
"Ref": "AWS::AccountId" | ||
}, | ||
":rule/StepFunctionsGetEventsForStepFunctionsExecutionRule" | ||
] | ||
] | ||
} | ||
} | ||
], | ||
"Version": "2012-10-17" | ||
}, | ||
"PolicyName": "ParentRoleDefaultPolicy9BDC56DC", | ||
"Roles": [ | ||
{ | ||
"Ref": "ParentRole5F0C366C" | ||
} | ||
] | ||
} | ||
}, | ||
"Parent8B210403": { | ||
"Type": "AWS::StepFunctions::StateMachine", | ||
"Properties": { | ||
"DefinitionString": { | ||
"Fn::Join": [ | ||
"", | ||
[ | ||
"{\"StartAt\":\"Task\",\"States\":{\"Task\":{\"End\":true,\"Type\":\"Task\",\"Resource\":\"arn:", | ||
{ | ||
"Ref": "AWS::Partition" | ||
}, | ||
":states:::states:startExecution.sync:2\",\"Parameters\":{\"Input\":{\"hello.$\":\"$.hello\"},\"StateMachineArn\":\"", | ||
{ | ||
"Ref": "ChildDAB30558" | ||
}, | ||
"\"}}}}" | ||
] | ||
] | ||
}, | ||
"RoleArn": { | ||
"Fn::GetAtt": ["ParentRole5F0C366C", "Arn"] | ||
} | ||
}, | ||
"DependsOn": ["ParentRoleDefaultPolicy9BDC56DC", "ParentRole5F0C366C"] | ||
} | ||
}, | ||
"Outputs": { | ||
"StateMachineARN": { | ||
"Value": { | ||
"Ref": "Parent8B210403" | ||
} | ||
} | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as with batch.
Can we call
new StepFunctionsStartExecution()
in the constructor here and delete most of the code in this file?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that's pretty natural and organic if we were replacing one API with another to create an adapter and call into the replacement. I don't think this is a scenario where we can just proxy into the replacement as the replacement contains and configures properties (i.e. inputPath, outputPath, ...) that the older
In this case, we are replacing a class that implemented a bind method to contribute a portion of the properties to represent a Task with a class that's a construct and configures all of the properties for a Task state.
The user experience with the current format will be to embed the call to start execution into the invocation of a new
Task
. I don't think proxying to invoke a new StepFunctionStartExeuction yields the right replacement.Some of the other ones to come like DynamoDB and ECS, I've tried to reuse the shared types. I could perhaps move more into a central location here and DRY it up a bit.
Did you have something else in mind? If so, are you opposed to me making a subsequent PR to do some refactoring?