Angular CLI and monorepos (like lerna, yarn workspaces)

[dherges]

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [x] feature request

Versions.

$ ng --version
    _                      _                 ____ _     ___
   / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
  / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
 / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
/_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
               |___/
@angular/cli: 1.3.0-beta.0
node: 6.10.1
os: darwin x64
@angular/animations: error
@angular/common: error
@angular/compiler: error
@angular/core: 4.3.1
@angular/forms: error
@angular/http: error
@angular/platform-browser: error
@angular/platform-browser-dynamic: error
@angular/router: error
@angular/cli: error
@angular/compiler-cli: error
@angular/language-service: error

This in itself manifests the error.

Repro steps.

A full repro can be found at GitHub repo spektrakel-blog/a-glimpse-at-yarn-workspaces

Initially, faced the "You seem to not be depending on @angular/core" error.
This goes down to a sanity check whether Angular is installed as a local dependency.

However, there are more errors with yarn workspaces, when dependencies from a workspace (or "sub-project", or "sub-package") are installed to the top-most node_modules directory of the workspaces root.

Then, it ends up w/ file structure:

Workspace root package.json:

  "workspaces": [
    "packages/*",
    "demo"
  ]

Demo workspace package.json:

{
  "name": "demo",
  "version": "0.0.0",
  "license": "MIT",
  "private": true,
  "dependencies": {
    "@angular/common": "^4.2.4",
    "@angular/core": "^4.2.4",
    "@angular/forms": "^4.2.4",
    "@angular/http": "^4.2.4"
  }
}

Now, dependencies are installed to node_modules and not to demo/node_modules:

$ ls -a demo/node_modules/
.               ..              .bin            .yarn-integrity
$ ls -a node_modules/@angular
.                        common                   forms                    platform-browser-dynamic
..                       compiler                 http                     router
animations               compiler-cli             language-service         tsc-wrapped
cli                      core                     platform-browser

Then, running ng build from the demo folder errors:

$ cd demo
$ yarn build
yarn build v0.27.5
$ ng build
You seem to not be depending on "@angular/core". This is an error.
error Command failed with exit code 2.

As a workaround, it's possible to symlink "@angular/core":

$ mkdir -p ./node_modules/@angular
$ ln -sf ../../../node_modules/@angular/core ./node_modules/@angular/core
$ ls -l node_modules/@angular/core
lrwxr-xr-x  ... node_modules/@angular/core -> ../../../node_modules/@angular/core
$ cat node_modules/@angular/core/package.json
{
  "name": "@angular/core",
  "version": "4.3.1",
  "description": "Angular - the core framework",
  "main": "./bundles/core.umd.js",
  ...
}

But then we only get one error further until:

$ yarn build
yarn build v0.27.5
$ ng build
Hash: a90d0476b33232a953c1
Time: 53409ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {3} [initial] [rendered]
chunk    {1} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {3} [initial] [rendered]
chunk    {2} main.bundle.js, main.bundle.js.map (main) 1.89 MB [initial] [rendered]
chunk    {3} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

ERROR in Error encountered resolving symbol values statically. Function calls are not supported. Consider replacing thefunction or lambda with a reference to an exported function (position 194:50 in the original .ts file), resolving symbol NgModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/core/core.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts

Which suggests that Angular CLI isn't really meant to work w/ "sub-projects" like yarn worksapces.

The log given by the failure.

See above.

Desired functionality.

Angular CLI and webpack should resolve depndencies from "top-level" node_modules.
The sanity check should honor node's module resolution algorithm (looking up recursively up the file tree).

Mention any other details that might be useful.

Related to #6504

A related change was implemented in #6475 (resolve in all available node_modules), but it seems that it does not address this use case.

[dherges]

At this point, this is me primarily asking:

"How is Angular CLI meant to work with monorepos?"

It the answer is "please use Angular CLI from the monorepo root directory", I can live by that.

[filipesilva]

I'm not really familiar with the Yarn workspaces, but the CLI currently attributes meaning to the location both both .angular-cli.json and package.json.

It expects them to be in the same dir, assumes that dir is the project root, and that there's also a node_modules there. These requirements are pretty pervasive throughout the CLI.

So no, I don't think the CLI is currently well equipped to work in monorepos.

But that's not because we don't want it to, just because it wasn't ever much of a concern. If there's a reasonable way of making it work I'm all for it really.

I think some of the CLI users are using it with Lerna, but I don't know the details of it.

The important bits of monorepo support is that it's not really specific to any single setup (like Lerna or these Yarn workspaces), and that it doesn't compromise the current setup.

[dherges]

Regarding my original post: I need to check whether preserveSymlinks offers an easy solution / workaround. #7194 #7081

---

Regarding discussion:

What may happen in monorepos (at least in yarn workspaces) that they install "(workspace-)local" and "(project-)global" dependencies. Example:

|- aio
    |- node_modules
        |- @angular
            |- http # <- 4.3.2
    | package.json
|- node_modules
    |- @angular
        |- common # <- 4.3.2
| package.json
|- packages
   |- my-lib
       |- node_modules
           |- @angular
                |- common # <- 4.2.0
       | package.json

Let aside potential version conflicts (which are in the hand of the user).

The difficulty for the webpack build would be to resolve modules "by walking up the tree until it finds one". I remember from past custom webpack configs that you had to pass the location of the node_modules. Was it in resolve?

https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/models/webpack-configs/common.ts#L91

Is this even possible to resolve modules from different directories?

[filipesilva]

I think it's possible, yeah. But now that got me thinking how that happens with peer deps. In your example there's different versions of packages that really want the same version of the peer deps. So through node module resolution you'd end up getting the different versions.

But even though the module resolution might work, I think you'd get.... a static analysis error. Which is what you actually got initially:

$ yarn build
yarn build v0.27.5
$ ng build
Hash: a90d0476b33232a953c1
Time: 53409ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {3} [initial] [rendered]
chunk    {1} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {3} [initial] [rendered]
chunk    {2} main.bundle.js, main.bundle.js.map (main) 1.89 MB [initial] [rendered]
chunk    {3} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

ERROR in Error encountered resolving symbol values statically. Function calls are not supported. Consider replacing thefunction or lambda with a reference to an exported function (position 194:50 in the original .ts file), resolving symbol NgModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/core/core.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts, resolving symbol BrowserModule in /Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/platform-browser/platform-browser.d.ts

This wasn't just an error finding a module, it was an error finding a module while using static analysis to find lazy loaded modules (we do that on every build, not just AOT).

So I ask... what happens in your setup when you install all the @angular/* packages at a given level, satisfying the peerdeps? It might just work.

And if that works the only thing we need to do to support it is to relax the You seem to not be depending on "@angular/core". This is an error. one, by performing some better @angular/core checks (https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/upgrade/version.ts#L85-L115).

See where it tries to use path join with the project root? We should be able to use node module resolution instead like https://github.com/angular/angular-cli/blob/master/packages/%40angular/cli/utilities/require-project-module.ts.

[dherges]

@filipesilva Ack.

|- demo
    | .angular-cli.json
    |- node_modules
        |- @angular # symlink to  ../../node_modules/@angular
|- node_modules
    |- @angular

$ ng build
WARNING in ../~/@angular/compiler/@angular/compiler.es5.js
(Emitted value instead of an instance of Error) Cannot find source file 'compiler.es5.ts': Error: Can't resolve './compiler.es5.ts' in '/Users/David/Projects/github/spektrakel-blog/a-glimpse-at-yarn-workspaces/node_modules/@angular/compiler/@angular'
 @ ../~/@angular/platform-browser-dynamic/@angular/platform-browser-dynamic.es5.js 7:0-72
 @ ./src/main.ts
 @ multi ./src/main.ts

So still the same error.

$ ng build --preserve-symlinks
Your global Angular CLI version (1.3.0-rc.3) is greater than your local
version (1.3.0-beta.0). The local Angular CLI version is used.

To disable this warning use "ng set --global warnings.versionMismatch=false".
Hash: d97b10ba4761c0902221
Time: 19229ms
chunk    {0} polyfills.bundle.js, polyfills.bundle.js.map (polyfills) 177 kB {4} [initial] [rendered]
chunk    {1} main.bundle.js, main.bundle.js.map (main) 94.3 kB {3} [initial] [rendered]
chunk    {2} styles.bundle.js, styles.bundle.js.map (styles) 10.5 kB {4} [initial] [rendered]
chunk    {3} vendor.bundle.js, vendor.bundle.js.map (vendor) 1.8 MB [initial] [rendered]
chunk    {4} inline.bundle.js, inline.bundle.js.map (inline) 0 bytes [entry] [rendered]

So with --preserve-symlinks the build is fine!

Also AoT / Prod build!

$ ng build --preserve-symlinks --aot --prod
Your global Angular CLI version (1.3.0-rc.3) is greater than your local
version (1.3.0-beta.0). The local Angular CLI version is used.

To disable this warning use "ng set --global warnings.versionMismatch=false".
Hash: 31d53744fb697d9b9f87
Time: 19407ms
chunk    {0} polyfills.3b4be225e7f6a233ebb3.bundle.js (polyfills) 177 kB {4} [initial] [rendered]
chunk    {1} main.a772681d78c70ca6637d.bundle.js (main) 101 kB {3} [initial] [rendered]
chunk    {2} styles.d41d8cd98f00b204e980.bundle.css (styles) 69 bytes {4} [initial] [rendered]
chunk    {3} vendor.e839d82f8b5326dc3f41.bundle.js (vendor) 761 kB [initial] [rendered]
chunk    {4} inline.6eaed4d70fc63a0f7481.bundle.js (inline) 0 bytes [entry] [rendered]

[filipesilva]

Ok, so this is progress. I take it the whole partial dep thing still won't work too well (for peer deps at least) but at least stuff seems to work if we tell webpack to pretend symlinks aren't really there.

BTW --preserve-symlinks was the work of @clydin, so big thanks to him for enabling this.

[intellix]

This was merged recently in TS and is probably available in nightly so I think the symlinks won't be required anymore: microsoft/TypeScript#16274

Edit: this doesn't solve what I thought it would :(

[about-code]

Similar to feature request #6083