Cannot consume ES module based cypress config when using Yarn 3

[lmiller1990]

Current behavior

Does not work correctly with cypress.config.ts containing ES module syntax when using Yarn 3 (does not create node_modules). I get an error and cannot proceed.

I think the problem is we try to resolve your TS install from projectRoot but new versions of yarn install modules in some other place that isn't in <project>/node_modules.

There's a lot of information in this issue, and related ones - before working on this, spend a bit of time reading and understanding the issue. It looks like it is primarily related to Plug n Play feature.

Note: many users are excited for this fix - if you are working it, please build a pre-release binary and share it here, so users can test it out.

Test code to reproduce

https://github.com/lmiller1990/yarn-3-issue-ts

• clone
• make sure you get 3.x when you do yarn --version
• install
• open cypress CT

Cypress Version

10.3.0+

Other

Created from #22071 (comment)

[gajus]

Any chance that you've figured out a solution?

[lmiller1990]

I didn't try yet, but it should be as simple as pointing Cypress at where ever Yarn PNP puts node_modules.

Happy to assist with this if you'd like to either take a stab, or even just point me in the direction - I know Cypress well, but not Yarn 3.

[jefbarn]

@lmiller1990 Thanks for looking into this, I'm trying to switch a project over to Yarn PNP and cypress is my last hold up.

Yarn PNP uses a custom loader and virtual filesystem, so there is not a 'node_modules' to point to. The trick should be to make sure ts-node is using the yarn loader, which is usually accomplished when loading a node script using yarn command eg 'yarn cypress run'.

But not working in this case because I suspect the cypress command is spinning off another process to run ts-node?

I'm happy to help play around with this if you have some suggestions. Also, I tried renaming my cypress.config.ts file to cypress.config.mts, which should be supported by the latest ts-node, but cypress doesn't like the mts extension yet.

[lmiller1990]

Not sure where to start with this, I'll need to do some research.

I do not think we support cypress.config.mts yet - we may need to bump our ts-node version. So many extensions and module formats now 🤯

[lmiller1990]

• Where we grab the ts-node loaders https://github.com/cypress-io/cypress/blob/develop/packages/data-context/src/data/ProjectConfigIpc.ts#L19-L20
• Where we attempt to execute the child_process using the right loader: https://github.com/cypress-io/cypress/blob/develop/packages/data-context/src/data/ProjectConfigIpc.ts#L273-L309

Probably need to do something around here, not sure what. I think what might be happening is this line is erroneously false, since it is attempting to resolve the user's TypeScript version using the usual node module algorithm, which doesn't work with Yarn 3.

What would be interesting is to see what happens in a Yarn 3 project without TypeScript - just using regular CJS. If that works, my suspicion is probably correct? cc @rockindahizzy

[josephzidell]

Running into a similar issue. Using yarn 3.2.3, cypress 10.7.0.

[lmiller1990]

I think I was misleading slightly - the title should not be ES module based but TS based cypress config. I will update it.

Basically, yarn 3 + TS isn't working? I think it's because we try to load your local TS from node_modules, which is not a thing in Yarn 3... digging into the various module problems now (eg #22795) so hopefully can look into this, too.

[sarah96wang]

Running into the same error with Yarn 3.2.3 and TS during migration from Cypress 9 to 10. Any updates?

[lmiller1990]

I haven't had a chance to look at this in more depth yet. I'm sorry. I don't think I can look at this during Dec, but I would like to see this triaged and fixed in January. Let me try and bump this one up in priority.

[baus]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @lmiller1990 @marktnoonan @mike-plummer @rockindahizzy @warrensplayer @ZachJW34

[gusgard]

This is the workaround I found to run it with yarn v3 #22699 (comment)

[jefbarn]

@gusgard That workaround is not working in my environment. Are you using typescript for your cypress config?

[francisu]

@lmiller1990 - this might help

This remains an issue in Cypress 12.3.0

I have this as well. I think the problem is in the code of ProjectConfigIpc in forkConfigProcess where it launches a plain node instance to do things. As this node instance was not launched via yarn, it does not get the required patching to resolve packages according to the Yarn 2/3 (PNP) rules. If the NODE_OPTIONS were set to require the .pnp-cjs file then it should work. I have not been able to test this because I don't have time to figure out how to build a cypress distribution from source and try it.

I also wanted to try it by patching cypress as it was installed on my machine, but the cypress npm dist does not seem to contain this code.

I'm happy to help test any proposed solution. And if I can get help in patching the cypress dist as it's installed (via yarn), then I can make the patch and try it.

[francisu]

This is the workaround I found to run it with yarn v3 #22699 (comment)

This seems to only apply to the webpack-dev-server. Some (including me) are using vite.

[nemonemi]

I am experiencing the same issue.
New NX project with Yarn 3 PnP mode and Cypress@12.17.1.

I have tried converting it to .js or .mjs but nothing worked.

As far as I understand, Cypress is trying to reference the Typescript from the node_modules and it doesn't know that it should actually search in the .yarn/cache folder. Would this be a correct assessment?

Your configFile is invalid: /home/app/frontend/apps/qualification/cypress.config.ts

It threw an error when required, check the stack trace below:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /home/app/frontend/apps/qualification/cypress.config.ts
    at new NodeError (node:internal/errors:399:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:79:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:121:38)
    at defaultLoad (node:internal/modules/esm/load:81:20)
    at nextLoad (node:internal/modules/esm/loader:163:28)
    at ESMLoader.load (node:internal/modules/esm/loader:605:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)
    at new ModuleJob (node:internal/modules/esm/module_job:64:26)
    at #createModuleJob (node:internal/modules/esm/loader:480:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[lmiller1990]

I don't think that is the issue here

Your configFile is invalid: /home/app/frontend/apps/qualification/cypress.config.ts

It threw an error when required, check the stack trace below:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for 
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:79:11)
    at defaultGetFormat (node:internal/modules/esm/get_format:121:38)
    at defaultLoad (node:internal/modules/esm/load:81:20)

It is using the native ESM loader which means the file must be .js to my understanding.

What's the minimal setup to repro this, I can take a look. Are you doing "type": "module" in your package.json by any chance?

[kubijo]

Cypress@13 has just been released and this still seems to be an issue… I have tried ts and mjs extensions to no avail. Is there a chance that this will get revisited soon?

Edit:

I might have gotten a little ahead of myself here, as the following change allowed me to open cypress@13 without that "invalid config file" error.

[lmiller1990]

If you are using type: module you need to include file extensions for Node.js to resolve the files correctly

[wiegell]

@lmiller1990 i have two do-what-you-want days next week at work and thought about looking into this, do you have any more pointers at where to start or maybe some general advice about this part of the codebase? I have read the overall contributors guideline.

[lmiller1990]

Hi! I'm not working on Cypress anymore, but I can do my best to point you in the right direction.

1. Get the project running locally. Have you done this?
2. Once you do, yarn dev at the root to open "global" mode.
3. Pick a Yarn 3 project, and follow the errors...

The issue I got stuck on was either the symlinks for things like @packages/config wouldn't resolve for Yarn 3, or it might have been something else... I think it may even be impossible to solve, since Cypress is an Electron app and Yarn 3 isn't really designed to work with CLI tools that launch Electron apps.

[wiegell]

I actually think i got something working yesterday, will try and make a PR tomorrow, thanks for the advice anyway and gl with the new job!

[lmiller1990]

Sure, happy to give your code a test and review if you need it.

[wiegell]

My progress so far:

To read the config.ts alone seems to be isolated to a single addition of the pnp node api:
https://github.com/cypress-io/cypress/pull/28134/files#r1370879639

But i have run into many other problems when trying to serve an angular app via the webpack-dev-server under pnp. See the error here:
https://github.com/cypress-io/cypress/pull/28134/files#r1370888179

I mostly run this command when trying to debug and then run the yarn-v3.1.1-pnp e2e test:

cd npm/webpack-dev-server && yarn build && cd ../.. && UPDATE_LOCK_FILE=1 DEBUG=cypress:lifecycle:ProjectConfigIpc,cypress:lifecycle:ProjectConfigManager,cypress:lifecycle:child:run_require_async_child:*,cypress:webpack-dev-server:angularHandler yarn workspace @packages/launchpad cypress:open

I have also tried experimenting with "type": "module", but that just opens up another box of errors...

Any general advice or thoughts are welcome @lmiller1990. It took half the day to just be able to navigate around the project - it's quite a beast :) I have some time again tomorrow to look into this.

[MikeMcC399]

It probably doesn't make any difference here, but the stable version of Yarn Modern is now v4 instead of v3 see https://yarnpkg.com/blog/release/4.0.

[vire]

It probably doesn't make any difference here, but the stable version of Yarn Modern is now v4 instead of v3 see https://yarnpkg.com/blog/release/4.0.

I've tried weith yarn4, cypress@13 has an issue with resolving files fron node_modules, cypress@12.17.3 has the following issue

require$$0.isBuiltin is not a function
TypeError: require$$0.isBuiltin is not a function
    at Function._resolveFilename (/Users/project/.pnp.cjs:86123:20)
    at Module._load (node:internal/modules/cjs/loader:785:27)
    at c._load (node:electron/js2c/asar_bundle:5:13339)
    at Function.<anonymous> (evalmachine.<anonymous>:1:751514)
    at i._load (<embedded>:4574:29304)
    at Function._load (/Users/project/.pnp.cjs:86086:31)
    at Module.require (node:internal/modules/cjs/loader:1012:19)
    at i (/Users/vire/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/index.js:1:2232)
    at customRequire (<embedded>:4633:1067438)
    at a (<embedded>:3108:259186)
    at q (<embedded>:3108:263120)
    at Object.A (<embedded>:3108:262786)
    at Object.INVALID_REPORTER_NAME (<embedded>:3567:107)
    at x (<embedded>:4105:226)
    at Object.w (<embedded>:4105:736)
    at C.initializeReporter (<embedded>:4626:388113)
    at C.startWebsockets (<embedded>:4626:388339)
    at C.open (<embedded>:4626:386668)
    at async v.create (<embedded>:4626:399192)
    at async O (<embedded>:4626:476208)
    at async W (<embedded>:4626:489449)

With yarn@3.6.4 no issues

[vire]

☝️ I have a NX17 + yarn4 + cypress3 repro https://github.com/vire/nx-17-yarn4-cypress13-repro

just git clone, run yarn, and then yarn nx run webapp-e2e:e2e

what you get is

Error: Webpack Compilation Error
Module not found: Error: Can't resolve 'uuid' in '/Users/vire/nx-17-yarn4-cypress13-repro/test-helpers/src/lib'

[elizww]

☝️ I have a NX17 + yarn4 + cypress3 repro https://github.com/vire/nx-17-yarn4-cypress13-repro

just git clone, run yarn, and then yarn nx run webapp-e2e:e2e

what you get is

Error: Webpack Compilation Error
Module not found: Error: Can't resolve 'uuid' in '/Users/vire/nx-17-yarn4-cypress13-repro/test-helpers/src/lib'

Same for me (different modules, failed to find react and ramda)

• NX 17, Cypress 13 with yarn 3.6.3

[MikeMcC399]

@elizww

• If you are using nodeLinker: pnp mode like @vire you may have more success if you convert to Yarn Modern nodeLinker: node-modules (also used by Yarn Classic) as a workaround.

• Cypress has multiple unresolved issues using Yarn Modern Plug'n'Play, especially for Component Testing, where it does not work at all. (See Yarn Modern Plug'n'Play incompatibility with CT missing in docs cypress-documentation#5394.)

[slikts]

@MikeMcC399 That's not a workaround, that's just not using Yarn PnP at all.

[MikeMcC399]

@slikts

That's not a workaround, that's just not using Yarn PnP at all.

Sorry if the suggestion was not helpful to you. According to other posts I have seen from @lmiller1990 , he has not been working for Cypress.io for several months now. I can't see any feedback in the issue from anybody else in the Cypress.io organization, so I don't know if there are plans to fix the Yarn Plug'n'Play support in Cypress.

If Yarn Classic projects are migrated to Yarn Modern they retain node-modules as nodeLinker parameter. Newly created Yarn Modern projects take pnp by default.

If you are using Yarn Modern and you are tied to using pnp by policy or for technical reasons, then I don't know of a workaround. My suggestion was only for those who might have the flexibility to use node-modules.

[slikts]

Again, the issue is not with anyone's flexibility, but with Cypress not supporting a Yarn feature that people go through the trouble of adopting because it's useful.

[MikeMcC399]

@slikts

Again, the issue is not with anyone's flexibility, but with Cypress not supporting a Yarn feature that people go through the trouble of adopting because it's useful.

I am also hoping that the Cypress.io team will fix this and other issues relating to Yarn Plug'n'Play!

[jinder]

Does the Cypress team have a plan to fix this? This issue has been open for years.