Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: feat(angular): add ivy support with JIT #11157

Closed
wants to merge 21 commits into from

Conversation

kroeder
Copy link
Member

@kroeder kroeder commented Jun 13, 2020

Issue: #10580 #10760 #10863 #13748 #13768

What I did

After the meeting with the Angular Team we figured out the missing piece of information that enables ivy support.

  1. In the webpack config: add mainFields for converted ivy packages
  resolve: {
      mainFields: [
        'es2015_ivy_ngcc',
        'module_ivy_ngcc',
        'main__ivy_ngcc',
        'es2015',
        'browser',
        'module',
        'main',
      ],
  }

If this is an environment without Ivy then it behaves the same as Storybook did before.

  1. Run ngcc
  • @angular/platform-browser-dynamic must be converted using ngcc
    • Targeting this package will also convert the dependencies of this package
  • We need a definite point in time to run ngcc
    • Angular does that either on post-install or during the startup of an Angular application
    • I'd suggest we also do this during the startup but with a check if ngcc is even executable in that environment in order to guarantee a fallback for ViewEngine environments
// ngcc cli command
ngcc --create-ivy-entry-points --target @angular/platform-browser-dynamic --first-only

How to test

tba

@github-actions
Copy link
Contributor

github-actions bot commented Jun 13, 2020

Fails
🚫

PR is not labeled with one of: ["cleanup","BREAKING CHANGE","feature request","bug","documentation","maintenance","dependencies","other"]

🚫 PR is marked with "do not merge" label.

Generated by 🚫 dangerJS against e506b81

@kroeder
Copy link
Member Author

kroeder commented Jun 13, 2020

If you start the example using this configuration all you get is

WARNING in D:\04_Development\storybook\examples\angular-cli\src\cssWarning.ts is part of the TypeScript compilation but it's unused.
Add only entry points to the 'files' or 'include' properties in your tsconfig.

for every *.ts file in the example/angular-cli directory.
Storybook pops up as expected but it could not load any stories, therefore no navigation and no preview and an endless spinning loading spinner

@kroeder kroeder mentioned this pull request Jun 24, 2020
@stupidawesome
Copy link

@kroeder I've created a tech demo showing what needs to be done to make CSF stories AoT compatible. I don't know how to integrate it with storybook, but its a start:

https://github.com/stupidawesome/storybook-csf-angular-aot-demo

If this is the approach we take there needs to be discussion on what syntax should be supported for writing stories since it has to be statically analyzable to work.

@shilman
Copy link
Member

shilman commented Jun 28, 2020

@stupidawesome can you elaborate on statically analyzable? I’m familiar with what that means in general from a compiler standpoint, but what does it mean specifically for AoT? Reference?

EDIT: checked out your repo and the examples look reasonable to me. I’m pretty busy with 6.0 release right now but when that’s further along I’d love to go through this together and help get it integrated!

@stupidawesome
Copy link

What I mean by statically analyzable is that which can be read/transformed/extracted by traversing the AST of the source files for each story. In practice it means that we expect identifiers and expressions to be in a certain format and location so that the parser can apply the right transforms to generate AoT compatible code (eg. arrow function/function declaration, parenthesized expression/return statement, top level variable statements etc). Anything outside of what is specified will either produce unexpected behavior,or simply fail to compile. For example: creating stories inside loop statements or arbitrary closures.

The requirements for making each story AoT compatible are:

  1. Generating a main.ts file with one call to platformBrowserDynamic.bootstrapModule()
  2. Generating NgModule and Component declarations from the CSF story file.
  3. Knowing how Angular's AoT compiler works.

Each story could then be compiled as a self contained microapp and served through the storybook canvas iframe. However AngularCompilerPlugin only supports a single entry point, so I'm not sure how to scale this to multiple stories right now.

@kroeder
Copy link
Member Author

kroeder commented Jul 6, 2020

@kroeder I've created a tech demo showing what needs to be done to make CSF stories AoT compatible. I don't know how to integrate it with storybook, but its a start:

https://github.com/stupidawesome/storybook-csf-angular-aot-demo

If this is the approach we take there needs to be discussion on what syntax should be supported for writing stories since it has to be statically analyzable to work.

@stupidawesome I finally could take a look at your repo. Amazing job! Thank you for that, it helped me a lot. 🙂

@shilman and I are going to chat about this on Thursday, 8AM (UTC)
Just give me a ping if you want to join the conversation!

Either reply here or join our discord https://discord.com/invite/UUt2PJb if you are interested 🙂

@stupidawesome
Copy link

@kroeder Can you make it 8:30? I've got a meeting to go to.

@kroeder
Copy link
Member Author

kroeder commented Jul 9, 2020

@stupidawesome sure! Can you join our discord so I can send you the zoom link later?

https://discord.com/invite/UUt2PJb
You can DM me using Kai_#5732

@stupidawesome
Copy link

stupidawesome commented Jul 14, 2020

It looks like we can enable Ivy while keeping things JIT with a few tweaks.

  • Setting skipCodeGeneration to true will allow JIT compilation with ɵivyEnabled printing true at runtime.
  • Angular 10 JIT mode expects decorators, meaning @Component or @NgModule can not be called as functions like Storybook currently does, so this just needs to change to use them as decorators on dynamic classes instead.
  • Angular elements works but seems redundant. Components can just be rendered with ApplicationRef or ComponentFactoryResolver then attached to the DOM dynamically and manipulating the native element as needed.

I think we should explore a way to compile user components with AoT separately while keeping the storybook metadata dynamic with JIT. That will hopefully avoid the need to do complicated TypeScript transforms while gaining the benefits of template type checking and dynamic storybook decorators.

@kroeder
Copy link
Member Author

kroeder commented Jul 26, 2020

@stupidawesome I pushed my changes
What I get is an empty storybook that fails to load any story file

WARNING in D:\04_Development\ng-projects\storybook\examples\angular-cli\src\stories\module-context\module-context.stories.ts is part of the TypeScript compilation but it's unused.
Add only entry points to the 'files' or 'include' properties in your tsconfig.

WARNING in D:\04_Development\ng-projects\storybook\examples\angular-cli\src\stories\on-push\on-push-box.component.ts is part of the TypeScript compilation but it's unused.
Add only entry points to the 'files' or 'include' properties in your tsconfig.

WARNING in D:\04_Development\ng-projects\storybook\examples\angular-cli\src\stories\on-push\on-push.stories.ts is part of the TypeScript compilation but it's unused.
Add only entry points to the 'files' or 'include' properties in your tsconfig.

But it should be part of the tsconfig file 🤔

@kossmoboleat
Copy link

Is this still being worked on? Soon one of our angular components will not be usable for us anymore because it's only published with AOT required.

@Marklb
Copy link
Member

Marklb commented Nov 12, 2020

@kossmoboleat I am attempting to make this or something more like the cli builders, but I haven't got something that would really work yet.

Since I had only used builders that hook into the cli for inserting webpack customizations, I have been looking into them and breaking down Angular's building code to try and understand enough to decide if AngularCompilerPlugin or something else would be best. The more I learn about their building code, the more annoyed I am getting with how it works. Hopefully their reasons for restricting the flexability of the compilation process will make sense as I learn more about it, but so far seems too opinionated. I don't have too much time to work on this though, so I may take a while to figure it out.

I am on Discord a lot, if anyone is working on this and wants to discuss it or has any questions I may can answer about Storybook's code.

@artaommahe
Copy link

@Marklb have you tried to discuss this stuff with angular/nrwl stuff members? You can find some of them in angular discord https://discord.gg/angular and maybe have some brainstorm to understand how to integrate aot build to stroybook. I think a lot of people there are interested in this

@Marklb
Copy link
Member

Marklb commented Nov 12, 2020

@artaommahe No, I haven't. I periodically go through their code or issues to check on questions people ask about Storybook, but haven't really discussed this with anyone. Now that I have a better understanding of the process, I may try to reach out to someone.

As for the Angular Discord, I watch the Storybook channel there and mentioned updating Storybook to support Ivy a few times, without any feedback. It wasn't necessarily questions I asked, so it isn't like I was ignored, just general comments that I am looking into it and would try to answer questions if anyone with more knowledge is looking into it.

@artaommahe
Copy link

artaommahe commented Nov 12, 2020

@Marklb mb point directly that you are trying to integrate aot stuff to storybook and need help? Also tag some people from angular/nrwl stuff and ask them to discuss this case or to get contacts to talk with, mb someone who works with angular-cli. Usually direct mentions work better than generic question

@artaommahe
Copy link

@Marklb just remembered, as far as i understand guys at jest-preset-angular have already done such integration of ivy thymikee/jest-preset-angular#409. Mb their solution can be applied here or to discuss this stuff with them

@ThibaudAV
Copy link
Contributor

@Marklb @artaommahe
I am not very familiar with webpack all the necessary concept for this PR🙈
I probably have a stupid question 😅:
Instead of integrate "angular build" in storybook we could not integrate sotrybook build in the one of angular ?
(cli-builder or other in web)
and be able to make a ng run storybook or something like that ?
It's not in my current competences. but I'm curious to get some feedback 🙃

@kroeder
Copy link
Member Author

kroeder commented Mar 1, 2021

Reminder: Include @angular/forms and @ngrx/* as ngcc target (internally)

import options from './options';

runNgcc();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a synchronous operation, correct?

Copy link
Member Author

@kroeder kroeder Mar 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can choose https://github.com/angular/angular/blob/master/packages/compiler-cli/ngcc/index.ts#L17

If I would provide AsyncNgccOptions as generic it would be async but as far as I can see it should be sync
I tested it in an app yesterday and it definitely was 🙂

Copy link

@petebacondarwin petebacondarwin Apr 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be aware that, in sync mode, ngcc will crash out if there is another ngcc process running at the same time - it is not able to wait like it can in async mode. This is probably not a problem but worth being aware of.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the info! I can imagine that someone starts the app + storybook in parallel right after installing all dependencies
Using async should cause no further issues, right? We just need to await ngcc

If it is that simple to prevent a crash then let's do this 🙂

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you can call runNgcc() asynchronously then it is probably safest to do so.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That error indicates that you are somehow triggering ngcc via its command line executable.
I don't suppose you have a prestorybook or prebuild script or something that is still running ngcc from the command line?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, only from the code. I extracted the code from this PR in a smaller external package which makes it easier to see what happens: https://github.com/storybookjs/addon-angular-ivy/blob/main/src/preset/index.ts

That's all

There are 2 ways to avoid the error

  • Use async: false
  • Use async: true and don't add arguments to the "start" script

No other prebuild steps involved: https://github.com/storybookjs/addon-angular-ivy/blob/main/package.json#L22

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I just realised... in async mode we kick off child processes, which will receive the arguments passed to the original parent process. Perhaps this is how they are getting passed to ngcc. I'll investigate

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A workaround would be to clear the command line args before triggering ngcc:

process.argv.length = 0;

@@ -205,7 +205,7 @@
"ts-dedent": "^2.0.0",
"ts-jest": "^26.4.4",
"ts-node": "^8.10.2",
"typescript": "^3.9.7",
"typescript": "^4.1.3",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool then we can proceed, with the upgrade I think!

@ndelangen
Copy link
Member

@kroeder looks like some TS errors came up since the upgrade?

@shilman
Copy link
Member

shilman commented Mar 10, 2021

Great work @kroeder ❤️

Let's merge this at the start of 6.3 so that we have lots of time to stabilize in case there are issues with Ivy, Typescript version, etc.

@ndelangen
Copy link
Member

I resolved merge cnflicts and synced the typescript version (cc @gaetanmaisse)

There's a bunch of test failing in the angular app @kroeder:

Summary of all failing tests
 FAIL  app/angular/src/client/preview/decorators.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/dev/Projects/GitHub/storybook/core/lib/addons/dist/cjs/public_api.js:4
    }});
    ^

    SyntaxError: Unexpected token '}'

    > 1 | import addons, { mockChannel, StoryContext } from '@storybook/addons';
        | ^
      2 |
      3 | import { Component } from '@angular/core';
      4 | import { moduleMetadata } from './decorators';

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1350:14)
      at Object.<anonymous> (src/client/preview/decorators.test.ts:1:1)

 FAIL  app/angular/src/server/__tests__/ts_config.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/dev/Projects/GitHub/storybook/core/__mocks__/fs.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){const fs = jest.data:application/json;charset=utf-8;base64,eyJmaWxlIjoiL1VzZXJzL2Rldi9Qcm9qZWN0cy9HaXRIdWIvc3Rvcnlib29rL2NvcmUvX19tb2Nrc19fL2ZzLmpzIiwic291cmNlcyI6WyIvVXNlcnMvZGV2L1Byb2plY3RzL0dpdEh1Yi9zdG9yeWJvb2svY29yZS9fX21vY2tzX18vZnMuanMiXX0=
                                                                                                                 ^

    SyntaxError: Unexpected token ':'

      2 |
      3 | // eslint-disable-next-line global-require, jest/no-mocks-import
    > 4 | jest.mock('fs', () => require('../../../../../__mocks__/fs'));
        |                       ^
      5 | jest.mock('path', () => ({
      6 |   resolve: () => 'tsconfig.json',
      7 | }));

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1350:14)
      at src/server/__tests__/ts_config.test.ts:4:23
      at Object.<anonymous> (src/server/ts_config.ts:1:1)

 FAIL  app/angular/src/server/__tests__/angular-cli_config.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/dev/Projects/GitHub/storybook/core/lib/node-logger/dist/cjs/index.js:4
    }});
    ^

    SyntaxError: Unexpected token '}'

      3 | import path from 'path';
      4 | import fs from 'fs';
    > 5 | import { logger } from '@storybook/node-logger';
        | ^
      6 | import { TsconfigPathsPlugin } from 'tsconfig-paths-webpack-plugin';
      7 | import stripJsonComments from 'strip-json-comments';
      8 | import {

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1350:14)
      at Object.<anonymous> (src/server/angular-cli_config.ts:5:1)

@Rush
Copy link

Rush commented Mar 15, 2021

I tried testing this by installing it with the commit id directly:

    "@storybook/angular": "storybookjs/storybook#d79bf0f84d3b816f356a03bb44a4231b894c103b",

Unfortunately looks like it's not as easy, I guess it needs a build step? Is there a version (tag?) published of this PR?

@ThibaudAV
Copy link
Contributor

ThibaudAV commented Mar 15, 2021

I guess it needs a build step?

@Rush Yes it is

Is there a version (tag?) published of this PR?

not to my knowledge


in some cases to test I use yarn link or a copy dist from app/angular
You just have to clone storybook project and run a yarn bootstrap --core

If you want to try

# Conflicts:
#	addons/storyshots/storyshots-core/package.json
#	yarn.lock
@ndelangen
Copy link
Member

running into this:

 FAIL  app/angular/src/server/__tests__/angular-cli_config.test.ts
  ● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/en/ecmascript-modules for how to enable it.
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    /Users/dev/Projects/GitHub/storybook/core/lib/node-logger/dist/cjs/index.js:4
    }});
    ^

    SyntaxError: Unexpected token '}'

      3 | import path from 'path';
      4 | import fs from 'fs';
    > 5 | import { logger } from '@storybook/node-logger';
        | ^
      6 | import { TsconfigPathsPlugin } from 'tsconfig-paths-webpack-plugin';
      7 | import stripJsonComments from 'strip-json-comments';
      8 | import {

      at Runtime.createScriptFromCode (../../node_modules/jest-runtime/build/index.js:1350:14)
      at Object.<anonymous> (src/server/angular-cli_config.ts:5:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.086 s
Ran all test suites matching /angular\/src\/server\/__tests__\/angular-cli_config.test.ts/i.

@kroeder
Copy link
Member Author

kroeder commented Mar 17, 2021

I have no clue why tests start to fail now. Those token errors are usually thrown when something is wrong in the compiler configuration 🤔 I haven't even changed anything in any tsconfig

Maybe this is caused by the typescript upgrade

@kroeder
Copy link
Member Author

kroeder commented Mar 17, 2021

I tried to rollback to typescript 3.9 but got the exact same error. I tried next and got no errors

@@ -1,4 +1,7 @@
module.exports = {
preset: 'jest-preset-angular',
setupFilesAfterEnv: ['<rootDir>/setup-jest.ts'],
moduleNameMapper: {
'@storybook/node-logger': '<rootDir>/../../lib/node-logger/dist/cjs/index.js',
Copy link
Member Author

@kroeder kroeder Mar 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly don't know how jest managed to successfully run tests with external packages without moduleNameMapper before 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now build-storybook fails with

ERR! Module parse failed: Unexpected character '@' (11:61)

I guess that's not the solution we were looking for

# Conflicts:
#	addons/storyshots/storyshots-core/package.json
@shilman
Copy link
Member

shilman commented Apr 29, 2021

Closing this for now. AFAIK this is being actively developed in https://github.com/storybookjs/addon-angular-ivy/. When that's ready we can bring that back into core. And if there is useful work in this PR, we can always restore it later.

@shilman shilman closed this Apr 29, 2021
@stof stof deleted the webpack-angular-compiler-plugin branch May 25, 2022 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.