Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): inflights are not required to be resources #4993

Merged
merged 43 commits into from
Nov 25, 2023
Merged

Conversation

MarkMcCulloh
Copy link
Contributor

@MarkMcCulloh MarkMcCulloh commented Nov 17, 2023

Huh?

The primary goal of this PR is to reduce the input required to create an inflight function in TS (See #4842) without necessarily overhauling the compiler (yet). Ideally, the minimum information required for an inflight is simply the code itself. However, because inflights are currently modeled as resources, they require a scope and id.

So the first change was to make a new non-resource interface, IInflight, encompassing the inflight contract. The most important part of this contract is that inflights must be liftable, a behavior currently unique to resources and certain other primitives. So I extracted the lift-related functions from IResource and slapped them on the new ILiftable (which both IInflight and IResource now extend).

But that created a new problem: lifting itself also currently requires a scope. The only usage of the scope was to be able to resolve tokens. This did not seem like a good enough reason to require scope, so I changed token resolution to be more of a global concern rather than a tree-level concern. This is dangerous, but it's mostly only dangerous when you try to deal with tokens in a multi-app scenario, which would be dangerous with our current approach anyways. So this is something we'll have to add some extra handling for eventually anyways.

Results

The primary outcome of this can be seen in the SDK unit tests, where the Testing.makeHandler() now only requires the code and (optional) bindings. This is basically 1 or 2 steps away from an ideal TS experience.

But wait nothing changed in winglang

The original purpose of representing inflights as resources was to ease the implementation of lifting in the compiler and generally unify the logic of inflights between inflight closures and inflight methods of preflight classes. This hasn't changed in this PR. Luckily, the class instance emitted by the wing compiler for inflights still satisfies IInflight. It just has some extra hidden resource stuff that is simply unused. Assuming this PR is wanted, I will do a followup to change the compiler as well. This will be a more complicated change and I think it's useful to basically get the backend working first.

Changes

  • Testing.makeHandler now takes only code text and bindings. 9 billion tests were updated for this contract. convertBetweenHandlers changed similarly
  • TokenResolvers are now globally registered and not tied to specific apps
  • wingc adds a _hash private field to inflight closure resource classes to match the new IInflight (just an md5 hash)
  • Many of the resources that deduped functions based on addr now do so with _hash instead
  • Removed many occurrences of this.node.id used in resource ids when it's not necessary. The only time this should be necessary is if the resources is being created in the scope of this.node.scope instead
  • Added a Counter concept to help with the many places that we want to add a subresource many times and a simple incrementing number will suffice for uniqueness and clarity
    • This was needed because the inflight addr was often relied upon to make this unique, but that will no longer be viable. I think it's better this way anyways

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

@monadabot
Copy link
Contributor

monadabot commented Nov 17, 2023

Console preview environment is available at https://wing-console-pr-4993.fly.dev 🚀

Last Updated (UTC) 2023-11-25 15:34

@monadabot
Copy link
Contributor

monadabot commented Nov 17, 2023

Benchmarks

Comparison to Baseline ⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜
Benchmark Before After Change
version 71ms±2.84 68ms±0.26 -3ms (-4.14%)⬜
functions_10.test.w -t sim 613ms±4.4 617ms±10.23 +4ms (+0.63%)⬜
functions_10.test.w -t tf-aws 3316ms±25.04 3324ms±26.73 +8ms (+0.25%)⬜
empty.test.w -t sim 514ms±3.53 521ms±6.77 +7ms (+1.29%)⬜
empty.test.w -t tf-aws 643ms±5.44 638ms±5.2 -5ms (-0.74%)⬜
hello_world.test.w -t sim 550ms±4.23 550ms±4.61 +0ms (+0.02%)⬜
hello_world.test.w -t tf-aws 4608ms±18.62 4584ms±15.74 -23ms (-0.5%)⬜
functions_1.test.w -t sim 551ms±6.82 554ms±5.53 +3ms (+0.58%)⬜
functions_1.test.w -t tf-aws 1538ms±24.22 1553ms±43.34 +15ms (+1%)⬜
jsii_big.test.w -t sim 3382ms±11.38 3396ms±21.3 +15ms (+0.43%)⬜
jsii_big.test.w -t tf-aws 3501ms±13.37 3491ms±9.35 -10ms (-0.29%)⬜
jsii_small.test.w -t sim 526ms±3.97 527ms±2.42 +1ms (+0.17%)⬜
jsii_small.test.w -t tf-aws 652ms±4.53 647ms±6.55 -5ms (-0.72%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results
name mean min max moe sd
version 68ms 67ms 69ms 0ms 0ms
functions_10.test.w -t sim 617ms 602ms 651ms 10ms 14ms
functions_10.test.w -t tf-aws 3324ms 3274ms 3375ms 27ms 37ms
empty.test.w -t sim 521ms 508ms 535ms 7ms 9ms
empty.test.w -t tf-aws 638ms 627ms 648ms 5ms 7ms
hello_world.test.w -t sim 550ms 538ms 558ms 5ms 6ms
hello_world.test.w -t tf-aws 4584ms 4553ms 4624ms 16ms 22ms
functions_1.test.w -t sim 554ms 540ms 566ms 6ms 8ms
functions_1.test.w -t tf-aws 1553ms 1430ms 1628ms 43ms 61ms
jsii_big.test.w -t sim 3396ms 3348ms 3438ms 21ms 30ms
jsii_big.test.w -t tf-aws 3491ms 3477ms 3521ms 9ms 13ms
jsii_small.test.w -t sim 527ms 523ms 534ms 2ms 3ms
jsii_small.test.w -t tf-aws 647ms 635ms 666ms 7ms 9ms
Last Updated (UTC) 2023-11-25 15:42

eladb
eladb previously approved these changes Nov 17, 2023
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a worthy direction to explore. May the gods of our unit test and type system guide you safely to a better place.

(Gut feeling is that the non determinism of inflight identifiers could bite us in the ass, but I am willing and p give this a go).

Also, please don't forget the "wing first" tenet. Eventually our typescript api needs to be produced based on wing declarations.

libs/wingsdk/src/core/tokens.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/core/tokens.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/shared-tf/app.ts Show resolved Hide resolved
libs/wingsdk/src/shared/misc.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/std/resource.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/std/resource.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/std/resource.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/std/resource.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/target-sim/bucket.inflight.ts Outdated Show resolved Hide resolved
@ekeren
Copy link

ekeren commented Nov 19, 2023

@MarkMcCulloh , is this suppose to resolve #2853

@MarkMcCulloh
Copy link
Contributor Author

@ekeren Not yet, although it's a step in that direction. This changes how the SDK works but the compiler will still be creating resources. This will need to change, but it is useful to change it here first (and it's backwards compatible)

@monadabot monadabot added the ⚠️ pr/review-mutation PR has been mutated and will not auto-merge. Clear this label if the changes look good! label Nov 21, 2023
@MarkMcCulloh MarkMcCulloh dismissed eladb’s stale review November 22, 2023 18:15

PR has underdone significant change

@MarkMcCulloh MarkMcCulloh requested a review from eladb November 22, 2023 18:15
@MarkMcCulloh MarkMcCulloh marked this pull request as ready for review November 22, 2023 18:15
@MarkMcCulloh MarkMcCulloh requested a review from a team as a code owner November 22, 2023 18:15
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few suggestions

libs/wingsdk/src/core/counter.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/shared/convert.ts Show resolved Hide resolved
libs/wingsdk/src/std/resource.ts Show resolved Hide resolved
libs/wingsdk/src/target-sim/bucket.inflight.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/target-tf-aws/queue.ts Show resolved Hide resolved
libs/wingsdk/src/core/counter.ts Outdated Show resolved Hide resolved
libs/wingsdk/src/core/counter.ts Outdated Show resolved Hide resolved
@MarkMcCulloh MarkMcCulloh requested a review from eladb November 23, 2023 14:44
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice and clean

Copy link
Contributor

mergify bot commented Nov 25, 2023

Thanks for contributing, @MarkMcCulloh! This PR will now be added to the merge queue, or immediately merged if mark/ts-fun is up-to-date with main and the queue is empty.

@mergify mergify bot merged commit edd91d4 into main Nov 25, 2023
@mergify mergify bot deleted the mark/ts-fun branch November 25, 2023 18:13
mergify bot pushed a commit that referenced this pull request Nov 25, 2023
…ct app (#5055)

Followup to #5054 (Accidentally changed sim instead of tf-aws)

Also follow up to #4993 (see topic.ts, reverted a change to call of `addPermissionToInvoke`)

Ran failing AWS tests locally this time to verify fix

*By submitting this pull request, I confirm that my contribution is made under the terms of the [Wing Cloud Contribution License](https://github.com/winglang/wing/blob/main/CONTRIBUTION_LICENSE.md)*.
@monadabot
Copy link
Contributor

Congrats! 🚀 This was released in Wing 0.50.3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants