Enhance modular-scripts Tests #2240

AlbertoBrusa · 2022-12-20T15:53:38Z

Standardize and clean up test utilities, such as all the different functions to run Modular in tests
Run tests in temp directories instead of within Modular repo itself
Don't let tests use git to stash changes etc
Limit use of yarn/installing packages etc to improve performance
Skip preflight check to improve performance
Refactor tests to simplify & reduce code duplication

Skipped tests because of Webpack issue

app.test.ts
app.node-env.test.ts
build.test.ts
index.test.ts

Removed Snapshots because of Webpack issue

esmView.test.ts | Removed snapshots of built entry file. if theres anything specific we want to check that the entry file contains, we could do something, but to me it looks like the tests still have good coverage and that the snapshots were unnecessary

changeset-bot · 2022-12-20T15:53:41Z

⚠️ No Changeset found

Latest commit: 88d1970

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

coveralls · 2022-12-20T16:20:34Z

Coverage: 25.546%. Remained the same when pulling 88d1970 on AlbertoBrusa:refactor-tests into aaaa5ee on jpmorganchase:feature/v4.

…into refactor-tests

…re deterministic? probably not

…terministic problem

… use it, deprecating runLocalModular

sgb-io

Looks good overall, just a few minor things I've commented on

sgb-io · 2023-01-13T10:09:46Z

packages/create-modular-react-app/src/__tests__/repo-integrity.test.ts

- ...opts,
- });
-}
+import { runModular } from 'modular-scripts/src/test/utils';


Is this import style intentional or required? should it be '../test/utils'?

the util file containing the function is in modular-scripts, while it's being called in create-modular-react-app

We don't have that dependency in package.json though (and we probably don't want to have it). We will probably factor out common utils when we do modularised Modular - since runModular is three lines anyway, can we just duplicate it here?

sgb-io · 2023-01-13T10:11:15Z

packages/modular-scripts/src/test/utils.ts

- * passing an array of `modularArguments`
- *
- * Useful to run modular from source in a different directory (usually a temp directory created for tests)
+ * Run the main repo's modular cli with the specified arguments, skipping modular checks by default to improve performance


Nit: when you say "main repo" here, what do you mean? Maybe it can be more explicit

This would be what we often refer to as ModularRoot - but I don't know if that makes it clearer, I'll try to rephrase it

sgb-io · 2023-01-13T10:13:50Z

packages/modular-scripts/src/test/utils.ts

 */
-export function runLocalModular(
- modularToRun: string,
+export function runModularUnsafe(


Nit: is this called Unsafe because of checks being skipped by default? If so, it took me a few minutes to grok that after looking at other functions in here, maybe it could just be explicitly stated. At a glance it's a bit subtle.

sgb-io · 2023-01-13T10:15:05Z

packages/modular-scripts/src/test/utils.ts

 },
 );
 }
+
+/**
+ * Wrapper of RunModularUnsafe that runs checks to make it safe & pipes output


Nit: related to my comment on runModularUnsafe; it wasn't obvious at first what makes this safe. I think it's because skipChecks is now false by default? Again, this could maybe just be clarified in a comment

packages/modular-scripts/src/test/utils.ts

* Node 18 & Jest 29 (#2210) * Add Node 18 to tests * Add Node 18 to supported engines * Pinned rollup-plugin-esbuild to 4.10.1 as newer versions drop support for node 14.17 and below * Updated yarn.lock * Update build.test snapshot * Update modular-scripts snapshot * Change rollup-plugin-eslint to latest compatible version * Added changeset * Update lockfile * Resolve to source-map ^0.7.0 to try and fix mozilla/source-map#432 * Try to resolve v8-to-istanbul to fix mozilla/source-map#432 * Dropped support for Node 14.17.0 (14.18.0 and greater) to allow upgrate to rollup-plugin-eslint 5 * Update tests to use Node 14.18.0 * Make tests use Node 14.18.0 * update jest to 29 * change Node 16 supported version to 16.10+ as that's what Jest supports * Upgrade to Jest 29 and fix problems caused by the upgrade * Update changelog * Update snapshots (default no longer includes escape characters etc) * Update snapshots not to include espace caracters * More snapshots updates * Change env for port tests to Node * Update snapshots and switch default test env to Node (while setting it to JSDOM for specific tests) * Update more snapshots * Remove broken Rename test (Rename functionality to be removed with Modular 4) * Fix tests (test.test wasn't closing after running with the --watchAll flag, and adding __fixtures__ to the lint exception list made tests fail) * Try to fix tests on Windows * DEBUG windows tests * Fix windows tests * Try more things to fix tests * More attempts to fix windows tests * More desperate attempts to fix windows tests * Clean up windows fix * Cleanup * Update changeset * Set Jest watchAll default to false regardless of if CI or not * Added ESLint ignore comments to svgr.ts for new TS complaints * Documentation * Update docs * Add feature/v4 to branches to run CI Tests on (Needs to be reverted before merging to main) * Small fixes * Refactor out generateJestConfig flag logic to reduce duplication * Update to ESLint 8 & Supported TypeScript to >4.5.3 (#2216) * Update ESLint to ^8.0.0, minimum Typescript to 4.5.3 & update dependencies * Update changeset * Fix warnings generated by updated ESLint * Remove commands and update documentation (#2220) * Remove commands and update documentation * Update Readme & Changeset * Merge main (#2225) Merge changes to modular 3.x from main since feature/v4 was split out * Use Config file instead of Environment Variables (#2227) * Use cosmiconfig to read modular configuration, while allowing Env variables to override it * Add configuration tests * Change the default CDN to esm.sh * Update documentation to reflect new configuration approach * Use INTERNAL_ env variables to read configuration in JS files that can't call the config function * Refactor config() logic Co-authored-by: Steve King <steve@mydev.co> * Add jsx to test glob pattern (#2234) * Add jsx to test glob pattern * Merge main into Modular 4 (#2237) * Merge main into Modular 34 * Sunset modular-site * Revert "Sunset modular-site" This reverts commit a27388b. * Sunset modular-site (#2238) * Sunset modular-site & update test snapshot * Remove requirement for Apps to be private in modualr check (#2241) * Update lockfile * Docs/support (#2248) * Initial cleanup * Add compatibility page, remove recipes/yarn * Better markdown section depth * update nav order * Document package types (#2246) * Start documenting package types * Describe App and ESM View types * Packages * Views, sources and templates * Link add command page * Link package types in index * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Link esm-views section in add page * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Link app section for more info * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Clarify manifest -> package.json + fix incorrect description * Add reasons why we don't support things * Disambiguate words * Phrase 'default export' concept better * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Reword tricky sentence * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Split long sentence * Add all package types to configuration docs * Various typos * Split packge types and move template page * Fix links to package types and fix table layout * Slim down table * build/start/entrypoint/template sections * Fix configuration * Remove list of types in documentation Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Generate nicer READMEs inside new projects / apps / views / packages (#2253) * Start documenting package types * Describe App and ESM View types * Packages * Views, sources and templates * Link add command page * Link package types in index * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Link esm-views section in add page * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Link app section for more info * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Clarify manifest -> package.json + fix incorrect description * Add reasons why we don't support things * Disambiguate words * Phrase 'default export' concept better * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Reword tricky sentence * Update docs/commands/add.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Update docs/concepts/package-types.md Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> * Split long sentence * Add all package types to configuration docs * Various typos * Split packge types and move template page * Fix links to package types and fix table layout * Slim down table * build/start/entrypoint/template sections * Add root project README * Add per-package READMEs + fix docs * Lint view documentation * Update snapshots * Upddate app snapshots * Update snapshots in index test * Update app.esbuild.test snapshots * update cmra tests * Correct snapshots for cmra * Update snapshots for cli.test.ts * Remove packages README + fix all READMEs * Add README to fixtures + update snapshots * Update various snapshots * Update app.esbuild.test snapshots * Add default workspace README * Update index snapshots * Create odd-bees-speak.md * Update fixture READMEs Co-authored-by: Sam Brown <sam.brown@jpmorgan.com> Co-authored-by: Alberto Brusa <alberto.brusa@live.com> * Sunset philosophy / views, write intro, fix web workers, correct templates (#2254) * Sunset philopsophy / views, move web workers, correct templates * Better front page * Fix command format * Add web workers docs * Enhance modular-scripts Tests (#2240) * Update tests in modular-scripts to use tmp directories instead of changing things within the repo * Consolidate all execa calls to yarn modular under one standardised function * Change calls to modular during tests to not run checks * Refactor tests for brevity, code reuse and raedability * Remove unnecessary build snapshots from esmVies.test * Modify getConfig to get workspace specific configuration (#2258) * Modify getConfig to get workspace specific configuration * Update config docs to reflect per workspace configuration * Document supported CRA features (#2255) * Rename limitations + start CRA page * Add CRA supported features * Fix typo * Fixes in CRA page * Test default template tests work (#2260) * Unify modular test command interface (#2259) * regex -> option, package -> argument * Windows test use --package * Remove space that fails test * Add debug statement * Implement selective testing and merging with user regexes * Update docs * Better wording * Fix typo * Test selective options combinations * modular build builds all packages * Test selective builds * Document behaviour of modular build * Create green-shrimps-build.md * Fix test docs * various docs fixes (#2262) * Small fixes (#2261) * Remove unnecessary error log * Move jest-environment-jsdom to correct package.json * Update typescript set by CMRA * More small fixes (#2263) * Remove unnecessary error log * Move jest-environment-jsdom to correct package.json * Remove jest-environment-jsdom from root package.json * reorder dependencies * v4.0.2 * v1.1.1 * v4.0.0 * Update typescript set by CMRA * Revert accidentally changed versions and bump CMRA version by major due to update TS version * Remove feature/v4 from GitHub Action tests & introduce forgotten CMRA changeset * Update 4.0.x release notes * Slight changes to release notes * Additions to release doc Co-authored-by: Cristiano Belloni <cristiano.belloni@jpmorgan.com> Co-authored-by: Cristiano Belloni <cristiano-belloni@users.noreply.github.com> Co-authored-by: Sam Brown <sam.brown@jpmorgan.com>

Update app.esbuild test

e33495b

AlbertoBrusa added 27 commits December 21, 2022 10:39

Update app.node-env tests

5b913d3

Add modular command that skips checks for tests

b6dfc82

FOR DEBUG: understand why built files are different on CI

afd8933

FOR DEBUG

8526534

FOR DEBUG

fa6fe68

FOR DEBUG

0b0caf2

FOR DEBUG

5e5b4f1

Update yarn lock

23f2072

FOR DEBUG

d58410d

FOR DEBUG - Revering to old test without temp directory

5b9b026

Merge branch 'feature/v4' of https://github.com/jpmorganchase/modular …

ab2a456

…into refactor-tests

Re-introduce new test after merging in feature/v4 changes

251115c

Set configuration webpack to deterministic

07aebb9

Try named module & chunk ids

b27a6af

Update esmView snapshot to use named chunk ids and module ids

cd22b9f

Update more snapshot for module & chunk id set to named

d20d926

Revert webpack build test to not use temp dir

df35a38

Update app.test & build.test to use modular without preflight checks

64634f7

Update snapshots

5ee5dc4

update snapshots

5dcfa37

Add cleanup back to app.node-env tests

165d58b

Refactor esmView tests

d1bf054

Fix puppeteer not working & browser not starting

a00a869

Remove debug message

09f8d6b

does including the Yarn.lock in the temp repo somehow make webpack mo…

351d03e

…re deterministic? probably not

Removing snapshots from esmView.test.ts failing due to Webpack non-de…

ff3df75

…terministic problem

Fix esmView.test

fa5ee7e

AlbertoBrusa added 5 commits January 10, 2023 09:40

Remove commented out tests from esmView.test.ts

35c65f5

Refactor esmView test

7761aa6

More test refactoring

ea8d896

More refactoring of esmView.test.ts

4039f8a

Update lint.test.ts

22d4254

AlbertoBrusa force-pushed the refactor-tests branch from 20a5af6 to 22d4254 Compare January 11, 2023 14:39

AlbertoBrusa added 11 commits January 11, 2023 17:17

Rename & increase scrope of runModularStreamlined + refactor tests to…

6839da3

… use it, deprecating runLocalModular

Fix tests

56052f7

Run test.test in temp directory

c955fa2

Further refactor test.test

9253c9e

Fix addPackage test

5cce253

Update sourceType.test.ts to use new runModular funcions

6cb5bd6

Improve typecheck.test.ts

0c1b564

Further clean up lint.test.ts

620cd3e

Update config.test.ts to use standardized Modular commands

b932753

Update getAllFiles.test.ts to use tmp directories

0b00680

Remove duplicate modular() function

0bd6e00

AlbertoBrusa marked this pull request as ready for review January 12, 2023 16:17

AlbertoBrusa changed the title ~~Enhance Tests~~ Enhance modular-scripts Tests Jan 12, 2023

Clean up app.test.ts

fb10a73

AlbertoBrusa requested a review from cristiano-belloni January 12, 2023 16:40

sgb-io approved these changes Jan 13, 2023

View reviewed changes

Rename run Modular functions and remove CMRA dep on modular-scripts

88d1970

sgb-io approved these changes Jan 13, 2023

View reviewed changes

cristiano-belloni approved these changes Jan 13, 2023

View reviewed changes

AlbertoBrusa merged commit 8c2c6f3 into jpmorganchase:feature/v4 Jan 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance modular-scripts Tests #2240

Enhance modular-scripts Tests #2240

AlbertoBrusa commented Dec 20, 2022 •

edited

Loading

changeset-bot bot commented Dec 20, 2022 •

edited

Loading

coveralls commented Dec 20, 2022 •

edited

Loading

sgb-io left a comment

sgb-io Jan 13, 2023

AlbertoBrusa Jan 13, 2023

cristiano-belloni Jan 13, 2023

sgb-io Jan 13, 2023

AlbertoBrusa Jan 13, 2023 •

edited

Loading

sgb-io Jan 13, 2023

sgb-io Jan 13, 2023

Enhance modular-scripts Tests #2240

Enhance modular-scripts Tests #2240

Conversation

AlbertoBrusa commented Dec 20, 2022 • edited Loading

changeset-bot bot commented Dec 20, 2022 • edited Loading

⚠️ No Changeset found

coveralls commented Dec 20, 2022 • edited Loading

sgb-io left a comment

Choose a reason for hiding this comment

sgb-io Jan 13, 2023

Choose a reason for hiding this comment

AlbertoBrusa Jan 13, 2023

Choose a reason for hiding this comment

cristiano-belloni Jan 13, 2023

Choose a reason for hiding this comment

sgb-io Jan 13, 2023

Choose a reason for hiding this comment

AlbertoBrusa Jan 13, 2023 • edited Loading

Choose a reason for hiding this comment

sgb-io Jan 13, 2023

Choose a reason for hiding this comment

sgb-io Jan 13, 2023

Choose a reason for hiding this comment

AlbertoBrusa commented Dec 20, 2022 •

edited

Loading

changeset-bot bot commented Dec 20, 2022 •

edited

Loading

coveralls commented Dec 20, 2022 •

edited

Loading

AlbertoBrusa Jan 13, 2023 •

edited

Loading