simple queue handler / cloud function takes longer than expected

[skorfmann]

I tried this:

bring cloud;
bring util;

let bucket = new cloud.Bucket();
let queue = new cloud.Queue();

queue.setConsumer(inflight (message) => {
  bucket.put("wing.txt", "Hello, ${message}");
}, timeout: 3s);

test "Hello, world!" {
  queue.push("world!");

  let found = util.waitUntil(() => {
    log("Checking if wing.txt exists");
    return bucket.exists("wing.txt");
  });

  assert("Hello, world!" == bucket.get("wing.txt"));
}

see example

wing test -t tf-aws main.w

This happened:

wing test --no-clean -t tf-aws main.w                                                                                                           <aws:monada-examples>
✔ Compiling main.w to tf-aws...

✔ terraform init

✔ terraform apply

✔ Setting up test runner...

✔ Running tests...

fail ┌ main.tfaws » root/Test.hiAsf5bKhU/env0/test:Hello, world!
     │ Error: Invoke failed with message: "Unhandled". Full error: "{"errorMessage":"2023-09-09T12:22:38.143Z 44cc9698-e6b2-436b-9691-016c01f6396c Task timed out after 30.07 seconds"}"
     │     at FunctionClient.invokeWithLogs (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/node_modules/@winglang/sdk/lib/shared-aws/function.inflight.js:50:19)
     │     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     │     at async TestRunnerClient.runTest (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/node_modules/@winglang/sdk/lib/shared-aws/test-runner.inflight.js:28:41)
     │     at async /Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/commands/test.js:314:30
     │     at async withSpinner (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/util.js:55:24)
     │     at async testTfAws (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/commands/test.js:311:25)
     │     at async testOne (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/commands/test.js:127:20)
     │     at async testFile (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/commands/test.js:55:39)
     │     at async Promise.all (index 0)
     │     at async Object.test (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/commands/test.js:66:5)
     └     at async Command.<anonymous> (/Users/sebastian/.nvm/versions/node/v20.6.0/lib/node_modules/winglang/dist/cli.js:46:30)

The queue handler is timing out - it actually takes almost 10 seconds for this function to succeed (see comment below)

I expected this:

It should not time out, it did work two weeks ago

Is there a workaround?

increase the timeout / add more resources to the function

Component

SDK

Wing Version

0.29.11

Node.js Version

20.6.0

Platform(s)

MacOS

Anything else?

No response

Community Notes

• Please vote by adding a 👍 reaction to the issue to help us prioritize.
• If you are interested to work on this issue, please leave a comment.

[skorfmann]

actually not sure if it's the init part. To run this successfully it needs a timeout of more than 10 seconds.

REPORT RequestId: b9f009cc-2f4a-5fac-9446-54ba457679b9	Duration: 9206.72 ms	Billed Duration: 9207 ms	Memory Size: 128 MB	Max Memory Used: 100 MB	Init Duration: 173.75 ms
--

So, perhaps there's something else going on. But it's happening for a few days already, so I don't believe it's AWS related.

[skorfmann]

Seems like a regression of https://github.com/winglang/wing/releases/tag/v0.29.6 - so likely #4109

Running the example above

with 0.29.5

with 0.29.6

[staycoolcall911]

Nice catch @skorfmann

Related - I just had a short discussion with @eladb on #4109 where he pointed out something that passed under my radar;
By not including the aws-sdk in the lambda, we may run into versioning compatibility issues between the aws-sdk version preflight/inflight.
I'll open a separate issue on this.

[marciocadev]

Apparently, the embedded version of aws-sdk v3 in Node 18 is not well optimized.

There is a recommendation to deploy the Lambda with the necessary aws-sdk v3 objects. In general, Lambdas require specific modules (e.g., @aws-sdk/client-dynamodb).

My question is whether, during the bundling process, we are only including what is necessary or the entire library?

[skorfmann]

Esbuild should be bundling the necessary code parts only

[marciocadev]

then maybe its better bundling the aws-sdk v3 again, code will be urge but run faster

[marciocadev]

I did a quick test here, using Node.js 18 without adding the aws-sdk v3 library, using the one already available in Lambda.
The execution time is around 8.5 seconds.

When I upload the code with the aws-sdk v3 library, the execution time drops to 2.3 seconds.

@staycoolcall911 it's better to include the SDK in the object again

[staycoolcall911]

Thanks @marciocadev - I guess we should go back to inline the awssdk code in the lambda then.
This, however, brings us back to the original problem you set out to solve - the lambda has 46k lines, but that's a smaller problem, so we should definitely do as you recommended

[monadabot]

Congrats! 🚀 This was released in Wing 0.29.20.