updating CLI from 6.1.5 -> 7.0.3 indicates missing tsconfig files for ng serve / test (Karma)

[michahell]

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request

Command (mark with an x)

- [ ] new
- [x] build
- [x] serve
- [x] test
- [ ] e2e
- [ ] generate
- [ ] add
- [ ] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Versions

node: v9.8.0
npm: 6.4.1
ng: 7.0.3
os: macOS Mojave

Repro steps

In our Angular app we use a library consisting of only .ts files. these are mostly typings (so we should maybe use .d.ts as suffix, but without this compilation worked fine in Angular 6.1.5.
After upgrading, for 3 out of 36 .ts files we get errors looking like the one in the log section below.

After including the include property in our tsconfig.json file:

"include" : [
    "node_modules/@customTypings/**/*.ts",
    "src/main.ts",
    "src/polyfills.ts"
  ]

we get the same error for our main.ts and polyfills.ts files. After adding also these files, that fixes compilation for ng serve and ng build. For ng test (with Karma) however, we still get the same errors, even though our tsconfig.spec.json extends tsconfig.json.

Will try to come up with an MWE, WIP

The log given by the failure

ERROR in ./node_modules/@customTypings/api/lib/common.ts
Module build failed (from ./node_modules/@ngtools/webpack/src/index.js):
Error: /Users/michahell/project/node_modules/@customTypings/api/lib/common.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.
The missing file seems to be part of a third party library. TS files in published libraries are often a sign of a badly packaged library. Please open an issue in the library repository to alert its author and ask them to package the library using the Angular Package Format (https://goo.gl/jB3GVv).
    at AngularCompilerPlugin.getCompiledFile (/Users/michahell/project/node_modules/@ngtools/webpack/src/packages/ngtools/webpack/src/angular_compiler_plugin.ts:1028:15)
    at plugin.done.then (/Users/michahell/project/node_modules/@ngtools/webpack/src/packages/ngtools/webpack/src/loader.ts:49:29)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:118:7)
 @ ./src/app/components/conversation-message/conversation-message.component.spec.ts 13:15-44
 @ ./src sync \.spec\.ts$
 @ ./src/test.ts

Desired functionality

compilation succeeds as before in 6.1.7

[alan-agius4]

@michahell what was the original error that you experienced before adding the files in the include.

The latest error that you are experiencing is legit and is working as expected as libraries should not contain any TypeScript files and has been the case even for version 6.

[michahell]

@alan-agius4
We're only seeing variations on the exact same error pointing to other .ts files, even before I added the files to include.
In version 6 we did not even get warnings regarding this and everything worked fine.
What is the way to go? Should we make everything .d.ts definitions ?

[michahell]

And should I not be allowed to do whatever I want in my libraries? Why is it prohibited to import .ts files? They should be warnings, just like the current error is a warning the user.

[filipesilva]

@michahell you certainly can do whatever you want in your libraries, but it is important to understand the implications of shipping typescript files.

When you ship typescript files in a library, and import them from application typescript code, then the build system for your application is the one that will be building the library typescript files.

Configuration might be required for that to work, especially on the tsconfig file. It's obvious enough that TS syntax that was introduced on newer versions won't work on older versions. But Typescript does not follow semver, and can introduce breaking changes in minors. So there is no real guarantee that TS syntax will be valid on upcoming versions. E.g. that what is valid under 3.0 is also valid in 3.1.

So all in all if you ship TS in a library for consumers to build, then the consumers will have to build it using the same TS version as the library is developed in. This is often reasonable enough for internal libraries.

It's definitely not prohibited to import .ts files, but of you do import them then it really is up to you to configure TS in such a way that it can actually build them. This often includes adding it to the files/include arrays.

For most of CLI 1.x we had a bug where we didn't really pay attention to the files listed in the tsconfig and just compiled all TS files that we came across. That changed in 1.5 and we started emitting *.ts is not part of the compilation errors (#8284 (comment)).

That should have been in place throughout all 6.x releases. So I'm not really sure of why you're seeing errors in 7.x but not in 6.x.

I think there is a bug somewhere, perhaps an error that is being swallowed. To figure out what's happening we'll need a reproduction, which you seem to be working on already.

[michahell]

@filipesilva Thank you for your elaborate answer, this all makes sense. our use case is indeed usage as internal library. I'll look into reproducing this now

[michahell]

Here is a reproduction of what I'm experiencing.
steps:

• I've used the Angular CLI to generate a new project using ng new
• then I added our own internal library .ts package to test if I was still experiencing this issue, which I was.
• I removed our internal library package
• created a sample package inside node_modules called @sample-ts-lib with a sample exported enum and interface.
• I use the enum from the sample package in app.component like so: const mountain = MountainEnum.meru;
• when I run both npm start and npm test I get the error mentioned above.
• I then update tsconfig.json with the following include property:

"include" : [
    "node_modules/@sample-ts-lib/**/*.ts",
    "src/main.ts",
    "src/polyfills.ts"
  ],

(side question: why do I have to explicitly include main.ts and polyfill.ts here? is it for the same reasons? but then why specify this in angular.json in the architect.build section?)

• now, I don't get compile errors with npm start. I do get compile errors with npm test.
• lastly, I tried adding "node_modules/@sample-ts-lib/**/*.ts" to the include property of tsconfig.json to no avail.

ts-lib-repro.zip

[mko]

We are currently seeing a similar issue. After upgrading, we found that our application seems to work fine, but our local test infrastructure fails because of the same Module build failed error related to our internally installed modules, which are symlinked into place for local development.

@michahell Are you guys symlinking your custom typings into place or are you actually installing the files into node_modules? We got rid of the symlinking and as soon as we did, Webpack/Karma properly picked up the custom modules.

[michahell]

@mko Thanks for the feedback! We are really installing them as a package into node_modules. Are you also adding files with a .ts extension instead of .d.ts? if so, could you give a barebone example of your tsconfig + app.tsconfig and test.tsconfig ?

[filipesilva]

@michahell I'm trying to use your repro but I can't see the error you mentioned. This is what I did:

• download the .zip file you provided and extract it into the ts-lib-repro folder
• go inside the folder
• notice there's a node_modules folder with @sample-ts-lib inside, but nothing else
• there's a yarn.lock file so I run yarn
• run npm run build and see the error below:

kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro (master)
$ npm run build

> ts-lib-repro@0.0.0 build C:\Users\kamik\Downloads\ts-lib-repro
> ng build


Date: 2018-11-09T14:16:56.736Z
Hash: 09489c23bd86db5f3663
Time: 5618ms
chunk {main} main.js, main.js.map (main) 628 bytes [initial] [rendered]
chunk {polyfills} polyfills.js, polyfills.js.map (polyfills) 94.6 kB [initial] [rendered]
chunk {runtime} runtime.js, runtime.js.map (runtime) 6.22 kB [entry] [rendered]
chunk {styles} styles.js, styles.js.map (styles) 17 kB [initial] [rendered]

ERROR in src/app/app.component.ts(2,30): error TS2307: Cannot find module '@sample-ts-lib/sample-enum'.

• check node_modules again and see that @sample-ts-lib disappeared, probably during yarn
• copy node_modules/@sample-ts-lib again from the zip file
• run npm run build again, see the error below:

kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro (master)
$ npm run build

> ts-lib-repro@0.0.0 build C:\Users\kamik\Downloads\ts-lib-repro
> ng build


Date: 2018-11-09T14:17:44.134Z
Hash: a905fc4c0cedc6d3ed28
Time: 3430ms
chunk {main} main.js, main.js.map (main) 628 bytes [initial] [rendered]
chunk {polyfills} polyfills.js, polyfills.js.map (polyfills) 94.6 kB [initial] [rendered]
chunk {runtime} runtime.js, runtime.js.map (runtime) 6.22 kB [entry] [rendered]
chunk {styles} styles.js, styles.js.map (styles) 17 kB [initial] [rendered]

ERROR in No NgModule metadata found for 'AppModule'.

I cannot see the error you mentioned (The missing file seems to be part of a third party library. etc) so I cannot investigate.

Can you provide me with precise instructions that I can follow to see the error please? To make sure it is reproducible, try following the repro yourself by doing the instructions.

[michahell]

@filipesilva ok will see what is going wrong, but the steps you followed should be correct...

[michahell]

OK for some reason, a default Angular CLI project defaults to yarn for me, I don't know why, I just used npm for this MWE.
Here is the same project, without .git, .idea and yarn.lock folders and files.
the @sample-ts-lib is moved into the project root.

What I did to make it 'work':

1. unzip folder somwhere
2. npm i
3. move @sample-ts-lib into the node_modules folder
4. npm start (nothing goes wrong here)
5. npm test (something DOES go wrong here)

ts-lib-repro-nogit-noyarn.zip

[filipesilva]

@michahell following your steps this is what I see:

kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro-nogit-noyarn/ts-lib-repro2
$ npm i
npm WARN deprecated circular-json@0.5.9: CircularJSON is in maintenance only, flatted is its successor.

> node-sass@4.9.3 install C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\node-sass
> node scripts/install.js

Cached binary found at C:\Users\kamik\AppData\Roaming\npm-cache\node-sass\4.9.3\win32-x64-64_binding.node

> node-sass@4.9.3 postinstall C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\node-sass
> node scripts/build.js

Binary found at C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\node-sass\vendor\win32-x64-64\binding.node
Testing binary
Binary is fine
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.4 (node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.4: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

added 1095 packages from 1163 contributors and audited 39132 packages in 95.417s
found 0 vulnerabilities


kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro-nogit-noyarn/ts-lib-repro2
$ mv \@sample-ts-lib/ node_modules/

kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro-nogit-noyarn/ts-lib-repro2
$ npm start

> ts-lib-repro@0.0.0 start C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2
> ng serve

** Angular Live Development Server is listening on localhost:4200, open your browser on http://localhost:4200/ **

Date: 2018-11-14T09:26:14.712Z
Hash: a61de44769ba995d983a
Time: 3586ms
chunk {main} main.js, main.js.map (main) 1.92 kB [initial] [rendered]
chunk {polyfills} polyfills.js, polyfills.js.map (polyfills) 92.4 kB [initial] [rendered]
chunk {runtime} runtime.js, runtime.js.map (runtime) 6.08 kB [entry] [rendered]
chunk {styles} styles.js, styles.js.map (styles) 16.6 kB [initial] [rendered]
chunk {vendor} vendor.js, vendor.js.map (vendor) 318 kB [initial] [rendered]

ERROR in No NgModule metadata found for 'AppModule'.
i ｢wdm｣: Failed to compile.


kamik@RED-X1C6 MINGW64 ~/Downloads/ts-lib-repro-nogit-noyarn/ts-lib-repro2
$ npm test

> ts-lib-repro@0.0.0 test C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2
> ng test

 11% building modules 9/9 modules 0 active14 11 2018 09:26:31.238:WARN [karma]: No captured browser, open http://localhost:9876/
14 11 2018 09:26:31.249:INFO [karma]: Karma v3.0.0 server started at http://0.0.0.0:9876/
14 11 2018 09:26:31.250:INFO [launcher]: Launching browser Chrome with unlimited concurrency
14 11 2018 09:26:31.256:INFO [launcher]: Starting browser Chrome                                                                                       × ｢wdm｣: Hash: 8d1070eadef0f8342111Version: webpack 4.19.1
Time: 9489ms
Built at: 11/14/2018 9:26:35 AM
       Asset      Size     Chunks  Chunk Names
     main.js  35.1 KiB       main  main
polyfills.js   525 KiB  polyfills  polyfills
   styles.js  20.8 KiB     styles  styles
   vendor.js  8.21 MiB     vendor  vendor
Entrypoint main = vendor.js main.js
Entrypoint polyfills = polyfills.js
Entrypoint styles = vendor.js styles.js
[./node_modules/@angular-devkit/build-angular/src/angular-cli-files/models/jit-polyfills.js] 233 bytes {polyfills} [built]
[./node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/raw-css-loader.js!./node_modules/postcss-loader/src/index.js?!./node_modules/sass-loader/lib/loader.js?!./src/styles.scss] ./node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/raw-css-loader.js!./node_modules/postcss-loader/src??embedded!./node_modules/sass-loader/lib/loader.js??ref--14-3!./src/styles.scss 554 bytes {styles} [built]
[./node_modules/@angular/compiler/fesm5/compiler.js] 1.13 MiB {vendor} [built]
[./node_modules/@angular/core/fesm5/testing.js] 76.3 KiB {vendor} [built]
[0] multi ./src/polyfills.ts ./node_modules/@angular-devkit/build-angular/src/angular-cli-files/models/jit-polyfills.js 40 bytes {polyfills} [built]
[./node_modules/@angular/platform-browser-dynamic/fesm5/testing.js] 9.7 KiB {vendor} [built]
[1] multi ./src/styles.scss 28 bytes {styles} [built]
[./node_modules/@angular/platform-browser/fesm5/platform-browser.js] 100 KiB {vendor} [built]
[./node_modules/@angular/platform-browser/fesm5/testing.js] 7.13 KiB {vendor} [built]
[./node_modules/core-js/es7/reflect.js] 510 bytes {polyfills} [built]
[./node_modules/zone.js/dist/zone-testing.js] 67.4 KiB {vendor} [built]
[./src sync recursive \.spec\.ts$] ./src sync \.spec\.ts$ 192 bytes {main} [built]
[./src/polyfills.ts] 3.14 KiB {polyfills} [built]
[./src/styles.scss] 1.54 KiB {styles} [built]
[./src/test.ts] 599 bytes {main} [built]
    + 298 hidden modules

ERROR in ./node_modules/@sample-ts-lib/sample-enum.ts
Module build failed (from ./node_modules/@ngtools/webpack/src/index.js):
Error: C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\@sample-ts-lib\sample-enum.ts is missing from the TypeScript compilation. Please make sure it is in your tsconfig via the 'files' or 'include' property.
The missing file seems to be part of a third party library. TS files in published libraries are often a sign of a badly packaged library. Please open an issue in the library repository to alert its author and ask them to package the library using the Angular Package Format (https://goo.gl/jB3GVv).
    at AngularCompilerPlugin.getCompiledFile (C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\@ngtools\webpack\src\packages\ngtools\webpack\src\angular_compiler_plugin.ts:1028:15)
    at plugin.done.then (C:\Users\kamik\Downloads\ts-lib-repro-nogit-noyarn\ts-lib-repro2\node_modules\@ngtools\webpack\src\packages\ngtools\webpack\src\loader.ts:49:29)
    at process._tickCallback (internal/process/next_tick.js:68:7)
 @ ./src/app/app.component.ts 11:0-58 15:23-35
 @ ./src/app/app.component.spec.ts
 @ ./src sync \.spec\.ts$
 @ ./src/test.ts
14 11 2018 09:26:36.684:INFO [Chrome 70.0.3538 (Windows 10 0.0.0)]: Connected on socket dNQFEa1gFZfe8oshAAAA with id 57726295
Chrome 70.0.3538 (Windows 10 0.0.0): Executed 0 of 0 ERROR (0.017 secs / 0 secs)

I see the build failing on both npm start and npm test.

If you change your src/tsconfig.spec.json from

  "include": [
    "node_modules/@sample-ts-lib/**/*.ts",
    "**/*.spec.ts",
    "**/*.d.ts"
  ]

to

  "include": [
    "../node_modules/@sample-ts-lib/**/*.ts",
    "**/*.spec.ts",
    "**/*.d.ts"
  ]

Then npm test will now pass. The path in a tsconfig include section are relative to the file itself, and there's no node_modules inside src/ so that include was never found.

I'm not sure what the error on build is but since you say you're not seeing it yourself then I suppose it's some odd artifact of the repro.

[michahell]

I would also dismiss the npm start error as some weird repro artifact because I started a new Angular CLI project from scratch.
But why does the src/tsconfig.spec.json need to specify the include property again in the first place? I would expect it to inherit this from the extended tsconfig.json ?

Maybe this is exáctly the problem i.e. - it does inherit although now the path is incorrect, because of the one directory level difference.
I'll double check this and report back.

[filipesilva]

When a property (like include) is re-declared in a tsconfig that inherits from another tsconfig, it is overwritten. src/tsconfig.spec.json has different includes so they need to be redeclared.

[michahell]

ah... that makes sense. I'd expected they would have been merged so no duplicating things / having a harder time maintaining.

and indeed, that seems to fix it, and then I have to be able to change this in our project, too.
Thanks @filipesilva !

[michahell]

I just really cannot seem to get this working in our project, really weird. What I've done now is add these 3 separate TS files the Angular compiler keeps complaining about in tsconfig.spec.json as follows:

"files": [
    "../node_modules/@zivver/api/lib/common.ts",
    "../node_modules/@zivver/api/lib/contacts.ts",
    "../node_modules/@zivver/api/lib/drafts.ts",
    "test.ts",
    "polyfills.ts"
  ],

because adding the folder to the include property also doesn't work:

"include": [
    "../node_modules/@zivver/api/lib/*.ts",
]

I still don't know why this is needed. But at least we can now continue working with Angular 7... 🤷‍♂️

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}