Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Dynamic StepFunction Dynamic Batching with Map and Iterator #4137

Closed
1 of 2 tasks
yyolk opened this issue Sep 18, 2019 · 4 comments · Fixed by #4382
Closed
1 of 2 tasks

Support for Dynamic StepFunction Dynamic Batching with Map and Iterator #4137

yyolk opened this issue Sep 18, 2019 · 4 comments · Fixed by #4382
Assignees
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions feature-request A feature should be added or improved. in-progress This issue is being actively worked on.

Comments

@yyolk
Copy link

yyolk commented Sep 18, 2019

In regards to this blog post: https://aws.amazon.com/blogs/aws/new-step-functions-support-for-dynamic-parallelism/ - AWS StepFunctions now support Dynamic Parallelism (released publicly today AFAICT). Here is the relevant section in the state-languages spec: https://states-language.net/spec.html#map-state

Like Task, and Parallel, I believe this would be a new Construct, with an IChainable interface for Map and Iterator

Use Case

I would like to use this in an in-development project that's already taking advantage of aws-cdk to create a workflow with stepfunctions. I'm currently iterating on payloads that can be executed in parallel, dynamically. I've steered away from trying to manage the state around a StateMachineFragment, but could go down that path until aws-cdk includes the aformentioned constructs.

Proposed Solution

Implement Map and Iterator constructs. (More is likely needed).

Other

supporting links:

  • 👋 I may be able to implement this feature request
  • ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

@yyolk yyolk added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Sep 18, 2019
@SomayaB SomayaB self-assigned this Sep 19, 2019
@SomayaB SomayaB added the @aws-cdk/aws-stepfunctions Related to AWS StepFunctions label Sep 19, 2019
@SomayaB SomayaB assigned rix0rrr and unassigned SomayaB Sep 19, 2019
@SomayaB
Copy link
Contributor

SomayaB commented Sep 19, 2019

Duplicate created #4154.
@rhboyd Work in progress PR #4153

@SomayaB SomayaB added in-progress This issue is being actively worked on. and removed needs-triage This issue or PR still needs to be triaged. labels Sep 19, 2019
@yyolk
Copy link
Author

yyolk commented Sep 24, 2019

now refs #4145

@yyolk
Copy link
Author

yyolk commented Oct 4, 2019

i believe this can be closed with #4145 getting merged

@apalumbo
Copy link
Contributor

apalumbo commented Oct 5, 2019

#4145 Should not work as there is a problem in the validation, I have produced the issue with a unit test that performs a synth

I have a fix, PR will be created in an hour

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions feature-request A feature should be added or improved. in-progress This issue is being actively worked on.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants