Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): dynamodb tasks #6654

Merged
merged 4 commits into from
Apr 3, 2020
Merged

feat(stepfunctions-tasks): dynamodb tasks #6654

merged 4 commits into from
Apr 3, 2020

Conversation

ayush987goyal
Copy link
Contributor

@ayush987goyal ayush987goyal commented Mar 10, 2020

Description

Add implementation to call DynamoDB APIs from StepFunctions based on this doc.

Note:

  1. DynamoDB CDK constructs are not used because of cyclic dependency (see stepfunctions task - support dynamodb #6468 ). Used tableName as string instead.
  2. The legacy parameters in all the APIs are not supported (eg. AttributesToGet in GetItem)

closes #6468

Commit Message

feat(stepfunctions-tasks): dynamodb tasks (#6654)

Support for DynamoDB as StepFunctions tasks. Supports GetItem,
PutItem, DeleteItem and UpdateItem tasks.

closes #6468

End Commit Message


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 813b8f4
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: dadd96d
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@nija-at
Copy link
Contributor

nija-at commented Mar 12, 2020

Could you enumerate the constructs and types for me, that would be used from the dynamodb construct library, had the cyclic dependency not occur?

@ayush987goyal
Copy link
Contributor Author

The only construct that I think I would be using from dynamodb would be the Table. It would replace the tableName prop from all the APIs.

Copy link
Contributor

@nija-at nija-at left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments so far.

@aws aws deleted a comment from mergify bot Mar 26, 2020
@mergify mergify bot dismissed nija-at’s stale review March 27, 2020 07:42

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 4f9a95d
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 1c20d38
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 46ced0f
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 64c591e
  • Result: FAILED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 281e483
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

@nija-at nija-at left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This is starting to come together.

packages/@aws-cdk/aws-stepfunctions/README.md Show resolved Hide resolved
@mergify mergify bot dismissed nija-at’s stale review March 30, 2020 18:38

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 18eef4a
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

nija-at
nija-at previously requested changes Apr 1, 2020
*
* @param attr Attribute name
*/
public withAttribute(attr: string): DynamoProjectionExpression {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here around supporting tokens. Make sure this works correctly and a test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how we would properly handle this scenario with tokens since we are doing string concatenation here.
For:

projectionExpression: [
        new tasks.DynamoProjectionExpression()
          .withAttribute('Messages')
          .atIndex(1)
          .withAttribute(sfn.Data.stringAt('$.tagsKey')),
        new tasks.DynamoProjectionExpression().withAttribute(
          sfn.Data.stringAt('$.idKey')
        )
      ]

I am getting the following error:

Field references must be the entire string, cannot concatenate them (found 'Messages[1].${Token[tagsKey.19]},${Token[idKey.20]}')

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'Messages[1].${Token[tagsKey.19]},${Token[idKey.20]}'

This should be valid. Where is this error coming from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the complete stack trace of the above error

Field references must be the entire string, cannot concatenate them (found 'Messages[1].${Token[tagKey.21]},${Token[otherKey.22]}')

      197 | 
      198 |   if (jsonPathTokens.length > 0 && fragments.length > 1) {
    > 199 |     throw new Error(`Field references must be the entire string, cannot concatenate them (found '${x}')`);
          |           ^
      200 |   }
      201 |   if (jsonPathTokens.length > 0) {
      202 |     return jsonPathTokens[0].path;

      at jsonPathString (../aws-stepfunctions/lib/json-path.ts:199:11)
      at Object.renderString (../aws-stepfunctions/lib/json-path.ts:146:16)
      at recurseObject (../aws-stepfunctions/lib/json-path.ts:90:35)
      at Object.renderObject (../aws-stepfunctions/lib/json-path.ts:36:10)
      at Function.renderObject (../aws-stepfunctions/lib/fields.ts:111:12)
      at Task.toStateJson (../aws-stepfunctions/lib/states/task.ts:161:65)
      at Object.<anonymous> (test/call-dynamodb.test.ts:251:29)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure what's going on here. This is good enough for now.
This will hopefully get fixed up in a later PR.

What you have here is already going to benefit a ton of customers!

@mergify mergify bot dismissed nija-at’s stale review April 1, 2020 10:45

Pull request has been modified.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: 61952c9
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@nija-at nija-at changed the title feat(sfn-tasks): call DynamoDB APIs feat(stepfunctions-tasks): dynamodb tasks Apr 3, 2020
Copy link
Contributor

@nija-at nija-at left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great! Thanks for the awesome contribution @ayush987goyal.

*
* @param attr Attribute name
*/
public withAttribute(attr: string): DynamoProjectionExpression {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure what's going on here. This is good enough for now.
This will hopefully get fixed up in a later PR.

What you have here is already going to benefit a ton of customers!

@mergify
Copy link
Contributor

mergify bot commented Apr 3, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

1 similar comment
@mergify
Copy link
Contributor

mergify bot commented Apr 3, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: fc191f7
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Apr 3, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 435b66f into aws:master Apr 3, 2020
@ayush987goyal ayush987goyal deleted the pr/sfn-ddb branch April 3, 2020 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stepfunctions task - support dynamodb
3 participants