`performance.markResourceTiming is not a function` in Node 18.x on AWS Lambda #2110

jamietanna · 2023-05-06T20:45:02Z

Bug Description

When running the below linked code, the Node process crashes with an undici-related issue due to the markResourceTiming function being undefined, when it does appear to exist.

Reproducible By

I've started a reproduction case for https://gitlab.com/jamietanna/bug-nodejs18-undici-renovate but unfortunately it doesn't seem to be working (as I'm trying to reproduce from some proprietary code) but will work to get it updated.

Expected Behavior

No error.

Logs & Screenshots

{
    "errorType": "TypeError",
    "errorMessage": "performance.markResourceTiming is not a function",
    "stack": [
        "TypeError: performance.markResourceTiming is not a function",
        "    at markResourceTiming (node:internal/deps/undici/undici:10636:21)",
        "    at finalizeAndReportTiming (node:internal/deps/undici/undici:10632:7)",
        "    at Object.handleFetchDone [as processResponseEndOfBody] (node:internal/deps/undici/undici:10579:45)",
        "    at node:internal/deps/undici/undici:10895:44",
        "    at node:internal/process/task_queues:140:7",
        "    at AsyncResource.runInAsyncScope (node:async_hooks:204:9)",
        "    at AsyncResource.runMicrotask (node:internal/process/task_queues:137:8)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
    ]
}

Environment

AWS Lambda, node18.x runtime with NodeJS version v18.5.0

Additional context

My debugging so far, via the README of the reproduction repo:

Prerequisites to reproduce

Due to a requirement (from the original case that this issue appeared in) to use the system-installed version of Git, we need to use a container image.

Create a new Lambda function, using a container image
Task timeout: 5 minutes
Memory: 512MB
Set an environment variable:
- RENOVATE_TOKEN set to a public_repo classic GitHub Personal Access Token

A pre-built image can be found at:

docker pull public.ecr.aws/s8m6r7h6/jamietanna-bug-nodejs18-undici-renovate:latest

And can be re-published to a private ECR for use with the container image function.

Error

The error we see, when the Lambda is triggered, is:

{
    "errorType": "TypeError",
    "errorMessage": "performance.markResourceTiming is not a function",
    "stack": [
        "TypeError: performance.markResourceTiming is not a function",
        "    at markResourceTiming (node:internal/deps/undici/undici:10636:21)",
        "    at finalizeAndReportTiming (node:internal/deps/undici/undici:10632:7)",
        "    at Object.handleFetchDone [as processResponseEndOfBody] (node:internal/deps/undici/undici:10579:45)",
        "    at node:internal/deps/undici/undici:10895:44",
        "    at node:internal/process/task_queues:140:7",
        "    at AsyncResource.runInAsyncScope (node:async_hooks:204:9)",
        "    at AsyncResource.runMicrotask (node:internal/process/task_queues:137:8)",
        "    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
    ]
}

We can see that this corresponds to this function:

    function markResourceTiming(timingInfo, originalURL, initiatorType, globalThis2, cacheState) {
      if (nodeMajor >= 18 && nodeMinor >= 2) {
        performance.markResourceTiming(timingInfo, originalURL, initiatorType, globalThis2, cacheState);
      }
    }

Where performance is imported from perf_hooks:

    var { performance: performance2 } = require("perf_hooks");

Is it set?

Writing the following pure JavaScript to debug:

var { performance: performance2 } = require("perf_hooks");

console.log({ performance })
console.log({ performance2 })

console.log({ func: performance.markResourceTiming })
console.log({ func: performance2.markResourceTiming })

We can see the following output:

2023-05-06T19:58:46.012Z	undefined	INFO	{
  performance: Performance {
    nodeTiming: PerformanceNodeTiming {
      name: 'node',
      entryType: 'node',
      startTime: 0,
      duration: 140.55377300013788,
      nodeStart: 8.12042300007306,
      v8Start: 49.48309800005518,
      bootstrapComplete: 107.49898899998516,
      environment: 69.97605099994689,
      loopStart: 113.04220100003295,
      loopExit: -1,
      idleTime: 0.190256
    },
    timeOrigin: 1683403125870.896
  }
}
2023-05-06T19:58:46.013Z	undefined	INFO	{
  performance2: Performance {
    nodeTiming: PerformanceNodeTiming {
      name: 'node',
      entryType: 'node',
      startTime: 0,
      duration: 142.08210300002247,
      nodeStart: 8.12042300007306,
      v8Start: 49.48309800005518,
      bootstrapComplete: 107.49898899998516,
      environment: 69.97605099994689,
      loopStart: 113.04220100003295,
      loopExit: -1,
      idleTime: 0.190256
    },
    timeOrigin: 1683403125870.896
  }
}
2023-05-06T19:58:46.013Z	undefined	INFO	{ func: [Function: markResourceTiming] }
2023-05-06T19:58:46.013Z	undefined	INFO	{ func: [Function: markResourceTiming] }
2023-05-06T19:58:46.014Z	undefined	ERROR	Uncaught Exception 	{"errorType":"Runtime.HandlerNotFound","errorMessage":"index.handler is undefined or not exported","stack":["Runtime.HandlerNotFound: index.handler is undefined or not exported","    at UserFunction.js.module.exports.load (file:///var/runtime/index.mjs:1035:15)","    at async start (file:///var/runtime/index.mjs:1195:23)","    at async file:///var/runtime/index.mjs:1201:1"]}
2023-05-06T19:58:46.445Z	undefined	INFO	{
  performance: Performance {
    nodeTiming: PerformanceNodeTiming {
      name: 'node',
      entryType: 'node',
      startTime: 0,
      duration: 341.72253999998793,
      nodeStart: 1.8998930000234395,
      v8Start: 98.50267300009727,
      bootstrapComplete: 238.43771099997684,
      environment: 136.76918400009163,
      loopStart: 260.6421229999978,
      loopExit: -1,
      idleTime: 0.03305
    },
    timeOrigin: 1683403126102.882
  }
}
2023-05-06T19:58:46.446Z	undefined	INFO	{
  performance2: Performance {
    nodeTiming: PerformanceNodeTiming {
      name: 'node',
      entryType: 'node',
      startTime: 0,
      duration: 343.41519299987704,
      nodeStart: 1.8998930000234395,
      v8Start: 98.50267300009727,
      bootstrapComplete: 238.43771099997684,
      environment: 136.76918400009163,
      loopStart: 260.6421229999978,
      loopExit: -1,
      idleTime: 0.03305
    },
    timeOrigin: 1683403126102.882
  }
}
2023-05-06T19:58:46.446Z	undefined	INFO	{ func: [Function: markResourceTiming] }
2023-05-06T19:58:46.446Z	undefined	INFO	{ func: [Function: markResourceTiming] }

So it's unclear as to why we're seeing it not set.

When we're using TypeScript:

var { performance: performance2 } = require("perf_hooks");

export function debug(): void {
  console.log({ performance })
  console.log({ performance2 })

  console.log({ func: performance.markResourceTiming })
  console.log({ func: performance2.markResourceTiming })
}

Trying to build this, we see a compilation error:

npm run build


> build
> tsc

debug.ts:7:35 - error TS2339: Property 'markResourceTiming' does not exist on type 'Performance'.

7   console.log({ func: performance.markResourceTiming })
                                    ~~~~~~~~~~~~~~~~~~


Found 1 error in debug.ts:7

But we can get around it with:

 var { performance: performance2 } = require("perf_hooks");

 export function debug(): void {
   console.log({ performance })
   console.log({ performance2 })

-  console.log({ func: performance.markResourceTiming })
+  console.log({ func: (performance as any).markResourceTiming })
   console.log({ func: performance2.markResourceTiming })
 }

The text was updated successfully, but these errors were encountered:

KhafraDev · 2023-05-06T21:29:19Z

this seems like an issue with AWS? I'd like to know why it's happening before patching it though.

mcollina · 2023-05-07T07:35:59Z

The linked URL is asking me for a Gitlab username/password, is it private?

jamietanna · 2023-05-07T07:37:02Z

Woops sorry, just made that public now

mcollina · 2023-05-07T09:26:55Z

The link you pointed us to is from Node.js core. If that's the problem you are dealing with, there is way we can fix it as we have absolutely no control on what AWS Lambda ships. Ultimately, updating Node.js would fix this problem, but it's not our responsibility.

What we can do is fixing undici so it runs correctly on Node v18.5.0 (and add it to CI), but the bundled version on Node.js core would stay buggy.

jamietanna · 2023-05-07T09:33:23Z

Thanks Matteo, I appreciate that. Unfortunately from the stack trace I've seen it's unclear as to what triggers this code path, do you know if performance.markResourceTiming expected to run at the end of every invocation to fetch?

KhafraDev · 2023-05-07T13:51:12Z

It is expected, but it was added in node v18.2.0 so I'm not sure why it wouldn't work.

jamietanna · 2023-05-07T16:46:12Z

Interestingly, using the proprietary code, I get the same error when using a Docker image that is following aws/aws-lambda-nodejs-runtime-interface-client#53 for an official NodeJS Docker image, not just Amazon's version

The only other similar case I could find while searching around is golang/go#57516

It may be that Amazon's https://www.npmjs.com/package/aws-lambda-ric - which hasn't been updated since Node 16 - just may not work with the new performance code brought in as part of Node 18, and as this is a bit of a workaround, not a general usecase, it may explain why it's not been seen anywhere else.

jamietanna · 2023-05-07T17:46:11Z

I appear to have gotten around the issue by setting NODE_OPTIONS=--no-experimental-fetch, via https://aws.amazon.com/blogs/compute/node-js-18-x-runtime-now-available-in-aws-lambda/:

Experimental features

While still experimental, the global fetch API is available by default in Node.js 18. The API includes a fetch function, making fetch polyfills and third-party HTTP packages redundant.

Experimental features in Node.js can be enabled/disabled via the NODE_OPTIONS environment variable. For example, to stop the experimental fetch API you can create a Lambda environment variable NODE_OPTIONS and set the value to --no-experimental-fetch.

It's unclear as to whether this is introduced through the polyfills, for instance if the libraries I'm using can't use the polyfills, or if there's something else at play.

I'll keep this thread updated to see if this issue continues, and try and get a minimal reproduction.

robertkowalski · 2023-06-06T12:10:08Z

i have the same issue :) using undici.request workarounds the issue.

mcollina · 2023-06-10T05:41:54Z

There is literally nothing that can be done, as we do not control what version of Node.js AWS ships with Lambda.

jamietanna added the bug Something isn't working label May 6, 2023

mcollina closed this as not planned Won't fix, can't repro, duplicate, stale Jun 10, 2023

kettanaito mentioned this issue Nov 11, 2023

TypeError: performance.markResourceTiming is not a function mswjs/msw#1851

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`performance.markResourceTiming is not a function` in Node 18.x on AWS Lambda #2110

`performance.markResourceTiming is not a function` in Node 18.x on AWS Lambda #2110

jamietanna commented May 6, 2023

KhafraDev commented May 6, 2023

mcollina commented May 7, 2023

jamietanna commented May 7, 2023

mcollina commented May 7, 2023

jamietanna commented May 7, 2023

KhafraDev commented May 7, 2023

jamietanna commented May 7, 2023

jamietanna commented May 7, 2023 •

edited

Loading

robertkowalski commented Jun 6, 2023

mcollina commented Jun 10, 2023

performance.markResourceTiming is not a function in Node 18.x on AWS Lambda #2110

performance.markResourceTiming is not a function in Node 18.x on AWS Lambda #2110

Comments

jamietanna commented May 6, 2023

Bug Description

Reproducible By

Expected Behavior

Logs & Screenshots

Environment

Additional context

Prerequisites to reproduce

Error

Is it set?

KhafraDev commented May 6, 2023

mcollina commented May 7, 2023

jamietanna commented May 7, 2023

mcollina commented May 7, 2023

jamietanna commented May 7, 2023

KhafraDev commented May 7, 2023

jamietanna commented May 7, 2023

jamietanna commented May 7, 2023 • edited Loading

robertkowalski commented Jun 6, 2023

mcollina commented Jun 10, 2023

`performance.markResourceTiming is not a function` in Node 18.x on AWS Lambda #2110

`performance.markResourceTiming is not a function` in Node 18.x on AWS Lambda #2110

jamietanna commented May 7, 2023 •

edited

Loading