feat(step-functions): support and utilize new intrinsics by thantos · Pull Request #468 · sam-goodwin/functionless

thantos · 2022-09-01T06:49:58Z

Depends on #510

Closes #77 Closes #282 Closes #330 Closes #395

Note: new intrinsics are not supported by localstack yet.

netlify · 2022-09-01T06:50:04Z

✅ Deploy Preview for effortless-malabi-1c3e77 ready!

Name	Link
🔨 Latest commit	`2b7af54`
🔍 Latest deploy log	https://app.netlify.com/sites/effortless-malabi-1c3e77/deploys/6324bc9725725700099f8132
😎 Deploy Preview	https://deploy-preview-468--effortless-malabi-1c3e77.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

…nto sussman/sfn_new_intrinsics

sam-goodwin · 2022-09-01T07:29:56Z

src/step-function.ts

+   * @param
+   */
+  export const range = makeStepFunctionIntegration<
+    "SFN.Range",


Should this be States.Range? To match the intrinsic name?

Though I am not using the full names of the intrinsics. (ArrayRange => Range) because "array" is covered by the type.

src/step-function.ts

sam-goodwin · 2022-09-01T07:33:36Z

src/step-function.ts

+  >("SFN.Base64Encode", {
+    asl(call, context) {
+      const [data] = call.args;
+
+      if (!data) {
+        throw new SynthError(
+          ErrorCodes.Invalid_Input,
+          "Expected Base64 data argument to be provided."
+        );
+      }
+
+      return context.evalContext(call, ({ evalExprToJsonPathOrLiteral }) => {
+        const dataOut = evalExprToJsonPathOrLiteral(data.expr);
+
+        if (
+          ASLGraph.isLiteralValue(dataOut) &&
+          typeof dataOut.value !== "string"
+        ) {
+          throw new SynthError(
+            ErrorCodes.Invalid_Input,
+            "Expected base64Encode data argument to be a string or reference."
+          );
+        }
+        const temp = context.newHeapVariable();
+        return {
+          Type: "Pass",
+          Parameters: {
+            "out.$": `States.Base64Encode(${
+              ASLGraph.isJsonPath(dataOut)
+                ? dataOut.jsonPath
+                : `'${dataOut.value}'`
+            })`,
+          },
+          ResultPath: temp,
+          Next: ASLGraph.DeferNext,
+          output: {
+            jsonPath: `${temp}.out`,
+          },
+        };
+      });
+    },
+  });


I wonder if there is anyway to simplify these new intrinsic. They seem to have a lot of code in common. Thoughts?

Created a helper to assign a single parameter key and output it. Was applicable in many places.

src/step-function.ts

sam-goodwin · 2022-09-01T07:37:52Z

test/localstack.ts

-    credentials: clientConfig.credentials,
-  };
+    sdkProvider.sdkOptions = {
+      // @ts-ignore


this was existing code that was moved.

Might be assigning some not so public props :)

test/step-function.localstack.test.ts

sam-goodwin · 2022-09-01T07:38:39Z

test/step-function.localstack.test.ts

+    (parent) => {
+      return new StepFunction(parent, "sfn", async () => {
+        const odd = $SFN
+          .range(1, 11, 2)


Can we use variables here

Yes. Will add more to the test.

…rinsics

Signed-off-by: github-actions <github-actions@github.com>

sam-goodwin

Oops, forgot to send these comments through lol

src/asl/asl-graph.ts

src/asl/synth.ts

sam-goodwin · 2022-09-14T04:46:24Z

src/asl/synth.ts

-                ResultPath: `$.${errorVariableName}`,
-              },
+              ASLGraph.assignJsonPathOrIntrinsic(
+                `States.StringToJson($.${errorVariableName}.Cause)`,


Should this be using one of the ASLGrapj.intrinscStringToJson calls?

Yep! My plan was to sweep through and update the whole code base to use these helpers after this change. Tried to get all of the intrinsics added in this change, but the helpers came much later than some of the other changes. Will clean up.

Updated most of them now

src/asl/synth.ts

sam-goodwin · 2022-09-14T04:49:13Z

src/asl/synth.ts

+            // postfix
+            // assign left to heap
+            // mutate left
+            // assign left
+            // return heap


Not sure what to do with these comments.

ha, just me trying to figure out the logic...

sam-goodwin · 2022-09-14T04:50:04Z

src/asl/synth.ts

+            if (ASLGraph.isLiteralValue(left)) {
+              if (
+                ASLGraph.isLiteralString(left) ||
+                ASLGraph.isLiteralNumber(left)
+              ) {
+                return ASLGraph.isLiteralValue(right)
+                  ? {
+                      value:
+                        right.value &&
+                        typeof right.value === "object" &&
+                        left.value in right.value,
+                      containsJsonPath: false,
+                    }


Going to need some comments.

sam-goodwin · 2022-09-15T04:55:54Z

src/error-code.ts

+   * Step Functions Arithmetic Only Supports Integer
+   *
+   * Step Functions only supports integer addition via the `States.MathAdd`.
+   *
+   * ```ts
+   * new StepFunction(stack, "sfn", async (input: { a: number }) => {
+   *    return 1.5 + input.a;
+   * });
+   * ```
+   *
+   * If the above machine is given input: `{ a: 0.5 }`, the result will be `1`.
+   *
+   * Effectively resulting in: `Math.floor(1.5) + Math.floor(0.5)` => `1`


I think this can be described more simply.

Step Functions can only add integers. When adding floating point numbers, the service will floor each of the numbers before adding them.

1.5 + 1.5 = 2

sam-goodwin · 2022-09-15T04:57:49Z

src/validate.ts

+          return [
+            newError(
+              node,
+              ErrorCodes.Cannot_perform_arithmetic_or_bitwise_computations_on_variables_in_Step_Function


Did we update this error code with the new addition change?

the logic, forgot about the text.

test/localstack.ts

test/step-function.test.ts

…rinsics

Signed-off-by: github-actions <github-actions@github.com>

…nto sussman/sfn_new_intrinsics

…ionless/functionless into sussman/sfn_new_intrinsics

Signed-off-by: github-actions <github-actions@github.com>

…ionless/functionless into sussman/sfn_new_intrinsics

…functionless into sussman/build_perf

Signed-off-by: github-actions <github-actions@github.com>

sam-goodwin · 2022-09-16T18:05:07Z

Is this ready? Looks good to me

thantos · 2022-09-16T18:07:33Z

Missed the approve and it was witting on #510, will push soon.

thantos added 3 commits August 31, 2022 17:10

adding function shims

c3346d1

more shims

19077c5

sfn intrisics working, all tests pass on AWS

c612f31

thantos added the step-functions label Sep 1, 2022

Merge branch 'main' of https://github.com/functionless/functionless i…

6f4304e

…nto sussman/sfn_new_intrinsics

sam-goodwin reviewed Sep 1, 2022

View reviewed changes

thantos added 22 commits September 1, 2022 12:31

feedback, refactor, and more tests

cb6ee3d

feat: length

d0112d0

update slice to support dynamic values

031d007

includes

18d4950

getItem

343b157

Merge remote-tracking branch 'upstream/main' into sussman/sfn_new_int…

763db7a

…rinsics

feat: array access and array in using references

00b68f0

feat: allow array access

11ede67

chore: support testing on AWS

72b5684

update AWS Credentials role

b3ea278

projen...

d0f9907

permissions?

82577d6

write write write

e1f8829

test role

89ee895

use cdk deploy role to deploy

4db7090

use cdk deployer for cfn

69f988a

log bad things

2aa5fe8

dont' copy cred provider

d01b6b1

unique export

c3f35b6

working locally?

c453ad8

fail on no cdk cred

ae79355

maybe it doesn't see the env?

54a1a4b

sam-goodwin mentioned this pull request Sep 4, 2022

Step Functions: support items.length > 0 in Conditions #77

Closed

chore: self mutation

b6753e5

Signed-off-by: github-actions <github-actions@github.com>

thantos requested a review from sam-goodwin September 14, 2022 12:37

sam-goodwin reviewed Sep 15, 2022

View reviewed changes

sam-goodwin approved these changes Sep 15, 2022

View reviewed changes

thantos and others added 19 commits September 15, 2022 02:42

feedback

50e7f03

Merge remote-tracking branch 'upstream/main' into sussman/sfn_new_int…

3e37664

…rinsics

fix bugs I caused

a61f543

chore: self mutation

17d0c53

Signed-off-by: github-actions <github-actions@github.com>

more memory, no jest cache, no hotswap

a345562

Merge branch 'main' of https://github.com/functionless/functionless i…

a24af95

…nto sussman/sfn_new_intrinsics

Merge branch 'sussman/sfn_new_intrinsics' of https://github.com/funct…

093e724

…ionless/functionless into sussman/sfn_new_intrinsics

chore: self mutation

c7a83ff

Signed-off-by: github-actions <github-actions@github.com>

chore: split up tests, other build improvements

677f016

upgrade jest

e320fe3

upgrade ts-jest and types

ac0c2be

chore: self mutation

f412c26

Signed-off-by: github-actions <github-actions@github.com>

Merge branch 'sussman/build_perf' into sussman/sfn_new_intrinsics

1235f89

Merge branch 'sussman/sfn_new_intrinsics' of https://github.com/funct…

4ad14e8

…ionless/functionless into sussman/sfn_new_intrinsics

await validate

41a3e92

Merge branch 'sussman/build_perf' of https://github.com/functionless/…

51446b2

…functionless into sussman/build_perf

projen and CI

ed7644b

fix issue with validate change

da32bd9

Merge branch 'sussman/build_perf' into sussman/sfn_new_intrinsics

db43f82

thantos changed the base branch from main to sussman/build_perf September 16, 2022 15:43

chore: self mutation

66bf18c

Signed-off-by: github-actions <github-actions@github.com>

Base automatically changed from sussman/build_perf to main September 16, 2022 18:06

Merge branch 'main' into sussman/sfn_new_intrinsics

2b7af54

thantos merged commit 1c5030f into main Sep 16, 2022

Conversation

thantos commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for effortless-malabi-1c3e77 ready!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sam-goodwin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sam-goodwin commented Sep 16, 2022

Uh oh!

thantos commented Sep 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thantos commented Sep 1, 2022 •

edited

Loading

netlify bot commented Sep 1, 2022 •

edited

Loading