Enable Corepack before installing Node.js dependencies

[koistya]

Currently, deploying a Node.js app with Yarn v4 (specified in the package.json/packageManager field) fails with the following error:

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed with status: FAILURE and message: error This project's package.json defines "packageManager": "yarn@4.0.1". However the current global version of Yarn is 1.22.20.

Presence of the "packageManager" field indicates that the project is meant to be used with Corepack, a tool included by default with all official Node.js distributions starting from 16.9 and 14.19.
Corepack must currently be enabled by running corepack enable in your terminal. For more information, check out https://yarnpkg.com/corepack..

References

• https://yarnpkg.com/corepack
• https://nodejs.org/api/corepack.html

[devinrhode2]

@koistya , other than this, how has the switch to yarn v4 been overall? Have you deployed your v4 upgrade to prod?

[devinrhode2]

Seems like yarn v4 is still very much a wip yarnpkg/berry#3591

[koistya]

@devinrhode2 I've been using Yarn v2, v3, v4 with GCF for the last few years, it was working great. With some minor code tweaks required on my end, after a couple of braking changes in the GCF build infrastructure.

Yarn v4.0.1 has been released last month:

https://yarnpkg.com/blog/release/4.0

BTW, I see that Cloud Functions using Yarn v4 stopped working today with the following error message:

ERROR: (gcloud.functions.deploy) OperationError: code=3, message=Build failed with status: FAILURE and message: error This project's package.json defines "packageManager": "yarn@4.0.1". However the current global version of Yarn is 1.22.20.

Presence of the "packageManager" field indicates that the project is meant to be used with Corepack, a tool included by default with all official Node.js distributions starting from 16.9 and 14.19.
Corepack must currently be enabled by running corepack enable in your terminal. For more information, check out https://yarnpkg.com/corepack..

Q: Was there an update to Cloud Functions build infrastructure recently?

[anothermh]

I have the same problem with a Gen2 Cloud Function using Yarn v4 with corepack enable. The workaround for me was to run yarn set version stable --yarn-path and go back to shipping the yarn binary from the repository again.

I also had to make sure I had package.json set properly so that detectYarnVersion() in the buildpack for node would parse out the correct version to use in yarnURL, and same with RequestedNodejsVersion() for nodejs:

{
  "packageManager": "yarn@4.0.2",
  "engines": {
    "node": "20.x.x",
    "yarn": "4.0.2"
  }

I'm able to deploy Gen2 functions now but it would be nice if there were an opt-in way to have the buildpack run corepack enable. It's still an experimental feature but there is an open issue about moving it to stable that can be tracked.

[FezVrasta]

Was there any progress on this? Support for Corepack is really needed.

[koistya]

Deploying Node.js aps to Google Cloud Functions (GCF) started to fail last week with the following error:

Error: Required package missing from disk. If you keep your packages inside your repository then restarting the Node process may be enough. Otherwise, try to run an install first.
Missing package: @google-cloud/functions-framework@npm:3.3.0
Expected package location: /www-data-home/.yarn/berry/cache/@google-cloud-functions-framework-npm-3.3.0-ac0c32fb0c-10c0.zip/node_modules/@google-cloud/functions-framework/

Even though, the yarn install build step seems to be completed without issues:

Step #2 - "build": Running "yarn install --immutable"
Step #2 - "build": ➤ YN0000: · Yarn 4.2.2
Step #2 - "build": ➤ YN0000: ┌ Resolution step
Step #2 - "build": ➤ YN0000: └ Completed
Step #2 - "build": ➤ YN0000: ┌ Fetch step
Step #2 - "build": ➤ YN0013: │ 462 packages were added to the project (+ 184.03 MiB).
Step #2 - "build": ➤ YN0000: └ Completed in 7s 226ms
Step #2 - "build": ➤ YN0000: ┌ Link step
Step #2 - "build": ➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
Step #2 - "build": ➤ YN0007: │ protobufjs@npm:7.2.6 must be built because it never has been before or the last one failed
Step #2 - "build": ➤ YN0000: └ Completed in 0s 468ms
Step #2 - "build": ➤ YN0000: · Done with warnings in 7s 951ms

UPDATE: It seems that Yarn's global cache needs to be disabled now in order to deploy a Node.js/Yarn bundle to GCF.

yarn config set enableGlobalCache false

[Willibaur]

Deploying Node.js aps to Google Cloud Functions (GCF) started to fail last week with the following error:

Error: Required package missing from disk. If you keep your packages inside your repository then restarting the Node process may be enough. Otherwise, try to run an install first.
Missing package: @google-cloud/functions-framework@npm:3.3.0
Expected package location: /www-data-home/.yarn/berry/cache/@google-cloud-functions-framework-npm-3.3.0-ac0c32fb0c-10c0.zip/node_modules/@google-cloud/functions-framework/

Even though, the yarn install build step seems to be completed without issues:

Step #2 - "build": Running "yarn install --immutable"
Step #2 - "build": ➤ YN0000: · Yarn 4.2.2
Step #2 - "build": ➤ YN0000: ┌ Resolution step
Step #2 - "build": ➤ YN0000: └ Completed
Step #2 - "build": ➤ YN0000: ┌ Fetch step
Step #2 - "build": ➤ YN0013: │ 462 packages were added to the project (+ 184.03 MiB).
Step #2 - "build": ➤ YN0000: └ Completed in 7s 226ms
Step #2 - "build": ➤ YN0000: ┌ Link step
Step #2 - "build": ➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
Step #2 - "build": ➤ YN0007: │ protobufjs@npm:7.2.6 must be built because it never has been before or the last one failed
Step #2 - "build": ➤ YN0000: └ Completed in 0s 468ms
Step #2 - "build": ➤ YN0000: · Done with warnings in 7s 951ms

UPDATE: It seems that Yarn's global cache needs to be disabled now in order to deploy a Node.js/Yarn bundle to GCF.

yarn config set enableGlobalCache false

Where did you set that yarn config set enableGlobalCache false @koistya❓

[devinrhode2]

I wonder what corepack thinks is the latest version of yarn. Could be just based on npm, which would be v1. I'm just on my phone but its easy to init a new project with corepack and see what yarn version it gives. See https://nodejs.org/dist/latest-v22.x/docs/api/corepack.html#upgrading-the-global-versions

…

On Sun, May 19, 2024 at 11:23 AM William Bautista ***@***.***> wrote: Deploying *Node.js* aps to *Google Cloud Functions* (GCF) started to fail last week with the following error: Error: Required package missing from disk. If you keep your packages inside your repository then restarting the Node process may be enough. Otherwise, try to run an install first. Missing package: @***@***.***:3.3.0 Expected package location: ***@***.******@***.***/functions-framework/ Even though, the yarn install build step seems to be completed without issues: Step #2 - "build": Running "yarn install --immutable" Step #2 - "build": ➤ YN0000: · Yarn 4.2.2 Step #2 - "build": ➤ YN0000: ┌ Resolution step Step #2 - "build": ➤ YN0000: └ Completed Step #2 - "build": ➤ YN0000: ┌ Fetch step Step #2 - "build": ➤ YN0013: │ 462 packages were added to the project (+ 184.03 MiB). Step #2 - "build": ➤ YN0000: └ Completed in 7s 226ms Step #2 - "build": ➤ YN0000: ┌ Link step Step #2 - "build": ➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental Step #2 - "build": ➤ YN0007: │ ***@***.***:7.2.6 must be built because it never has been before or the last one failed Step #2 - "build": ➤ YN0000: └ Completed in 0s 468ms Step #2 - "build": ➤ YN0000: · Done with warnings in 7s 951ms UPDATE: It seems that Yarn's global cache needs to be disabled now in order to deploy a Node.js/Yarn bundle to GCF. yarn config set enableGlobalCache false Where did you set that yarn config set enableGlobalCache false @koistya <https://github.com/koistya>❓ — Reply to this email directly, view it on GitHub <#201 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEDZKDL53A6CTNBTX4IS2TZDDGXXAVCNFSM5VPPQTQKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRHEZDSMJXGI4Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[anothermh]

I wonder what corepack thinks is the latest version of yarn. Could be just based on npm, which would be v1. I'm just on my phone but its easy to init a new project with corepack and see what yarn version it gives. See https://nodejs.org/dist/latest-v22.x/docs/api/corepack.html#upgrading-the-global-versions

It is the latest 1.x version:

$ docker run --rm -it --entrypoint sh node:20-alpine
/ # yarn -v
1.22.19
/ # corepack enable
/ # yarn -v
! Corepack is about to download https://registry.yarnpkg.com/yarn/-/yarn-1.22.22.tgz
? Do you want to continue? [Y/n]

1.22.22

But keep in mind this from the Corepack documentation:

The Corepack proxies will find the closest package.json file in your current directory hierarchy to extract its "packageManager" property.

If the value corresponds to a supported package manager, Corepack will make sure that all calls to the relevant binaries are run against the requested version, downloading it on demand if needed, and aborting if it cannot be successfully retrieved.

Therefore a properly formatted package.json will fetch the desired version:

$ docker run --rm -it --entrypoint sh node:20-alpine
/ # yarn -v
1.22.19
/ # cat << EOF > package.json
> {
>   "packageManager": "yarn@4.0.2",
>   "engines": {
>     "node": "20.x.x",
>     "yarn": "4.0.2"
>   }
> }
> EOF
/ # corepack enable
/ # yarn -v
! Corepack is about to download https://repo.yarnpkg.com/4.0.2/packages/yarnpkg-cli/bin/yarn.js
? Do you want to continue? [Y/n]

4.0.2

[phmcder]

Has a feature been raised to enable corepack as a step in the node buildpack or can I add that step myself somewhere?