Using yarn workspaces leads to wrong behavior of some subsystems

[otbe]

This is a (Bug Report / Feature Proposal) (maybe both?)

Description

I use serverless-webpack in a monorepo with several services. My structure is something like this:

├── packages/
│   ├── service1
│   └── service2
├── package.json
├── yarn.lock

Each service has is own package.json, serverless.yml and webpack.config.js.
This works so far but there are 2 issues with this setup:

1. I get a lot of WARNING: Could not check for peer dependencies of <dep> warnings. I guess this is related to this which just looks at the wrong place for node_modules. In webpack-node-externals I can define a folder it will look into, but that feels wrong too. Maybe we need a node-like resolution algo here?

2. If I'm using the yarn packager ,serverless-webpack will not find my lockfile because its some folders upwards in the tree.

I'm using the latest serverless/serverless-webpack version.

I really wonder if I should use serverless-webpackin this way and If so how?

Thank you for all your great work in this field 👍

[aneilbaboo]

I have a similar monorepo structure, where I have multiple subfolders each with their own serverless.yml and webpack configs.

I'm seeing a strange issue on CircleCI. When I first run a webpack build, it seems to ignore directives in the webpack configuration file, and instead attempts to transpile files where were explicitly excluded, including files from other subdirectories. The excluded files contain issues which cause webpack to faill.

The weird thing is that running the command again (i.e., yarn sls webpack) succeeds (i.e., on the second run, the excluded files don't trigger errors). Again, this only happens in my CI environment. But both serverless, webpack and serverless-webpack are the same in both my local and CI environments.

I thought perhaps some state was being generated in either .serverless/ or .webpack/ which caused webpack to ignore the problem files on the second run, but deleting these doesn't cause the problem to reappear.

I'm not sure if this is related to the issue you're seeing.

[esetnik]

I have the same issue using serverless-webpack with yarn workspaces. In my case it appears that only a warning is generated WARNING: Could not check for peer dependencies of <dep>

[sthuber90]

I observe the same output, and yarn.lockfile is not found in the project root, which for me is a not wanted behavior

[sthuber90]

I found a solution working well for me, using serverless-jetpack and serverless-scriptable-plugin

[j0k3r]

Can someone checkout the master and check the new option nodeModulesRelativeDir from #689 to tell me if it fix that issue? Thanks 🙏

[j0k3r]

Removed milestone until someone confirm my last comment

[zirkelc]

I think nodeModulesRelativeDir: '../../' resolves the issue with the modules, but it seems there is still another issue with yarn workspaces: `Error: yarn install --non-interactive --frozen-lockfile failed with code 1``

Could it be that the yarn.lock file can't be found in the root? However, even with noFrozenLockfile it still crashes.

[dls314]

I had some success using yarn's nohoist option universally, but the yarn.lock is still not found since it's in the root. I'm not sure if it will work, but I'm going to try copying the lock file to the serverless package directory prior to packaging.

[dls314]

I had some success using yarn's nohoist option universally, but the yarn.lock is still not found since it's in the root. I'm not sure if it will work, but I'm going to try copying the lock file to the serverless package directory prior to packaging.

This works for me, but it's a pretty hacky process. It would be nice if the yarn.lock location were configurable in the serverless.yml options in a similar way to the node_modules directory location.

[sambP]

I'm also getting this error Error: yarn install --non-interactive --frozen-lockfile failed with code 1. Moving the yarn.lock file into the serverless packages does not work for me. Does anybody have another idea how this could be resolved?

[rrapiteanu]

I'm also getting this error Error: yarn install --non-interactive --frozen-lockfile failed with code 1. Moving the yarn.lock file into the serverless packages does not work for me. Does anybody have another idea how this could be resolved?

Same for me, I am using turborepo and the yarn.lock + node_modules are outside of the serverless folder.

[sambP]

I got it to work with following settings:

serverless.yml

  webpack:
    webpackConfig: ./webpack-lambda.config.js
    packager: 'yarn'
    includeModules:
      nodeModulesRelativeDir: '../../'

webpack.config.js (all my packages are named @cr/*)

  externals: [
    nodeExternals({
      whitelist: [/^@cr/],
    }),
    nodeExternals({
      whitelist: [/^@cr/],
      modulesDir: path.resolve(__dirname, '../../node_modules'),
    }),
  ],

[dls314]

@sambP When you got it to work the way you showed, did you have to disable hoisting in the yarn workspace? Or is hoisting okay because you used the nodeModulesRelativeDir configuration?

I got it to work by copying the yarn.lock next to the serverless.yml and disabling hoisting in the yarn workspace, but doing that way took away some of the speed benefits of the yarn workspace.

[j0k3r]

Hey guys, @magelle improved the support for Yarn Workspace in #1258, can you try using the master of serverless-webpack before we cut a new release?