Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions): new service integration classes for Lambda, SNS, and SQS #7946

Merged
merged 25 commits into from
May 20, 2020

Conversation

shivlaks
Copy link
Contributor

@shivlaks shivlaks commented May 13, 2020

merge functionality currently modeled under task and state into a single object that represents a task state as an abstract class. Service integrations extend the base class and add service specific configuration, metrics, and policies.

this commit introduces the "new" service integrations for Lambda, SNS, and SQS

Motivation:
The current service integrations that are offered in aws-stepfunctions-tasks all currently implement a bind() method and contribute a portion of the configuration that creates a Task state.

However, it's often useful to configure state level properties such as paths, retries, errors based on the service integration and the pattern that has been requested.

Implementation:

  • Duplicate the current Task class and merge the properties of a task state and a task service integration into a new abstract base class.
  • Concrete implementation per service integration class where all of the best practices and user intents can be encoded
  • After all the service integrations have been migrated, we will want to retire the Task class as well since we don't want it to be instantiated.

Paves the way for:
#6715 by simplifying the invocation of service integration calls. We would be able to start adding properties such as retries and errors.

#6489 where we will be making service integrations consistent

End Commit Message


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@shivlaks shivlaks requested review from rix0rrr, nija-at and a team May 13, 2020 06:43
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label May 13, 2020
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: bf66763
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@rix0rrr
Copy link
Contributor

rix0rrr commented May 14, 2020

remove task token validation? (need feedback here)

Why?

@shivlaks
Copy link
Contributor Author

remove task token validation? (need feedback here)

Why?

I had a TODO in the comment, you addressed it though. Since users can insert the token anywhere in the payload (for Lambda), we can't ditch it.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: f58149c
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 6bf1eae
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: dfb89d4
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 375ae58
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 0cde795
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 193b048
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

shivlaks and others added 2 commits May 15, 2020 21:44

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Niranjan Jayakar <nija@amazon.com>
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: e87f1bf
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: dd07450
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@eikeon
Copy link

eikeon commented May 20, 2020

Looking good and looking forward to trying these out

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: d352300
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 28b7165
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 7183cb2
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@rix0rrr rix0rrr added the pr/do-not-merge This PR should not be merged at this time. label May 20, 2020
@rix0rrr
Copy link
Contributor

rix0rrr commented May 20, 2020

Don't forget a unit test on

      payload: sfn.TaskInput.fromDataAt('$.Payload')  

@shivlaks
Copy link
Contributor Author

Don't forget a unit test on

      payload: sfn.TaskInput.fromDataAt('$.Payload')  

it's here

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
Co-authored-by: Rico Huijbers <rix0rrr@gmail.com>
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: e40f1cc
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@shivlaks shivlaks changed the title feat(stepfunctions): new service integration classes that merge that task and state functionality feat(stepfunctions): new service integration classes for Lambda, SNS, and SQS May 20, 2020
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 6e695a8
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 76c5918
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@shivlaks shivlaks removed the pr/do-not-merge This PR should not be merged at this time. label May 20, 2020
@mergify
Copy link
Contributor

mergify bot commented May 20, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 9134c43
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented May 20, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
@mergify
Copy link
Contributor

mergify bot commented May 20, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit c038848 into master May 20, 2020
@mergify mergify bot deleted the shivlaks/sfn-merge-task-and-state-lambda branch May 20, 2020 22:29
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: fab0b69
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

karupanerura pushed a commit to karupanerura/aws-cdk that referenced this pull request May 22, 2020
… and SQS (aws#7946)

merge functionality currently modeled under task and state into a single object that represents a task state as an abstract class. Service integrations extend the base class and add service specific configuration, metrics, and policies.

this commit introduces the "new" service integrations for Lambda, SNS, and SQS

Motivation: 
The current service integrations that are offered in `aws-stepfunctions-tasks` all currently implement a `bind()` method and contribute a portion of the configuration that creates a `Task` state.

However, it's often useful to configure state level properties such as paths, retries, errors based on the service integration and the pattern that has been requested.

Implementation:
* Duplicate the current `Task` class and merge the properties of a task state and a task service integration into a new abstract base class.
* Concrete implementation per service integration class where all of the best practices and user intents can be encoded
* After all the service integrations have been migrated, we will want to retire the `Task` class as well since we don't want it to be instantiated.

Paves the way for:
aws#6715 by simplifying the invocation of service integration calls. We would be able to start adding properties such as retries and errors.

aws#6489 where we will be making service integrations consistent

### End Commit Message

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-stepfunctions Related to AWS StepFunctions @aws-cdk/aws-stepfunctions-tasks contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants