-
-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use import() on esm files in supported node versions #547
Conversation
Codecov Report
@@ Coverage Diff @@
## master #547 +/- ##
==========================================
- Coverage 74.65% 72.75% -1.91%
==========================================
Files 19 20 +1
Lines 789 800 +11
Branches 151 153 +2
==========================================
- Hits 589 582 -7
- Misses 200 218 +18
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that it's important these be serial - so when import()
is supported, i think the entire thing should be a .reduce
using Promises.
We also can’t upgrade tap due to the node versions it drops. |
i used a promise reducer to load test files serially and added failing tests. the new tests fail due to the problem i explained in the PR description, namely all tests in the files loaded after the first fail with
hmmm, ok. any other way of circumventing the instrumentation error? |
I'm not sure what the instrumentation error is; is it about |
regarding the failing tests, if you point me in the general direction of possible causes i am keen to start debugging and looking for solutions.
yes, the exact error is:
|
add |
also we'll need to make sure the tests that do imports are skipped in the appropriate node versions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lohfu what do you think about the following changes? I guess it will not work in node versions without ESM support, but still I’ve tried it on node 14.15.3 and seems to make the tests go green. If you can help me understand what I’m missing, maybe I can do the changes. Thank you so much.
@ljharb All tests pass now. If you are ok with the way i wait for all tests to load before running them I reckon the only things left now are:
|
Ok I did some very basic performance testing to see the implications of always using |
@ljharb what do you think about using this solution instead of checking which version of node is used to figure out if dynamic import is supported? |
@lohfu that's basically the solution i'll be using in the package i forgot to create :-) let me get on that today. |
Codecov Report
@@ Coverage Diff @@
## master #547 +/- ##
==========================================
+ Coverage 96.70% 96.77% +0.06%
==========================================
Files 4 4
Lines 607 620 +13
Branches 144 147 +3
==========================================
+ Hits 587 600 +13
Misses 20 20
Continue to review full report at Codecov.
|
@lohfu alrighty, give https://npmjs.com/has-dynamic-import a spin |
Sweet! Unfortunately I ran into some trouble with it immediately... I use has-dynamic-import to skip the ESM tests, and currently they arent skipped and fail in Node 10, 11, < 12.17 and < 13.2. All else seem to work fine. I created an issue and a PR explaining the issue and possible workaround. |
i have fiddled around a little with getting this PR working with inspect-js/has-dynamic-import#3. all tests seem to be working on my end, but i have the following questions:
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re 4, I think the t.plan(2)
has to happen synchronously, otherwise tap has no idea that those tests are incoming.
Re 1, I'm not a fan of having Results.__loadingPromise
exposed (see https://github.com/substack/tape/pull/547/files#r630197895)
Re 2, I'm not sure what you mean. "loaders" are already possible for CJS via require.extensions
.
Re 3, please feel free to squash/rebase as needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Re 4, I think the
t.plan(2)
has to happen synchronously, otherwise tap has no idea that those tests are incoming.
seemed to work fine, but i converted to using t.end()
just in case.
Re 1, I'm not a fan of having
Results.__loadingPromise
exposed (see https://github.com/substack/tape/pull/547/files#r630197895)
can't really think of any other solution... got any ideas?
Re 2, I'm not sure what you mean. "loaders" are already possible for CJS via
require.extensions
.
I was referring to ESM loaders... in the not too distant future i suspect some people want to import()
files beyond .mjs
or .js
.
When node supports something more, we can add support for it. At the moment, there are no other kinds of modules. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It occurred to me today that node only has --require
and does not have any way to evaluate ESM files on process load - it might be better to wait until node has that support before adding it here, in case they conflict.
eg, it's possible node will add --import=
for ESM, in which case that's what we'd want to have.
removed! |
I'm not sure what you removed - what i mean is, |
i removed the logic to use i am feeling pretty ready with this PR now... if you are ok with the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely getting close!
@@ -27,8 +27,10 @@ | |||
"defined": "^1.0.0", | |||
"dotignore": "^0.1.2", | |||
"for-each": "^0.3.3", | |||
"get-package-type": "^0.1.0", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m a bit concerned that this package’s engines declaration will cause false errors for yarn users, and anyone using engines-strict or https://npmjs.com/ls-engines, if it ever adds an engines declaration in the future (cc @coreyfarrell)
bin/tape
Outdated
} | ||
); | ||
} else { | ||
tape.run(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in a node without Promise, will this error just be surfaced to the process?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, that works. however, it apparently fails if Node supports a promise, but no import()
call has been made. since hasDynamicImport
creates a promise, we need to move the catch outside. I had to rethink the logic a bit... i'm beginning to feel ashamed 😞 ... but changed and pushed.
i should probably add some tests for this error handling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i've rebased the PR; please fetch before making further changes.
so i started using the changes in this branch on a project, and i've found a major drawback to the handling of rejections when loading test files. catching errors in our loading promise,
the actual stack trace contains no information of where the syntax error is; it's the lines before that discloses the error and its location. running the same file with this branch prints:
with only that information, it obviously becomes difficult to locate the problem without external tools. node appears to behave differently when the syntax errors are in commonjs files, ie when tape calls
|
Yeah, i think node does some magic with exceptions that are not caught and re-thrown. I’m not sure how to solve for that. |
After giving it some thought I have come up with some changes. The basic idea is to use After these last changes, this PR only affects people trying to load ESM files in versions that supports it and a loading promise will only be created if an ESM file is imported. This completely avoids unhandled rejections when using CommonJS and will only have awkward output in Node versions before 15. People converting their projects to full ESM most likely use very modern Node and as such, this should have poor behaviour for few. Moreover, 99% of unhandled rejections importing files will be syntax errors and developers have tools in place to locate those. On rejections importing ESM files, the previous changes printed::
Evidently this output gives very little clue as to were the error has occurred. As just mentioned, usually these errors will easily be solved with other developer tools, but in rarer cases not getting an exact location where the error was thrown can be struggling. After the last changes, promise rejections importing ESM files prints: Node >=15:
Node ^12.17, ^13.2 and 14:
All Node versions that do not support
For errors requiring CJS files, the output has always been the same both inside and outside promises (both caught and uncaught). |
It's not that they need to be handled; it's that module-level exceptions (at require/import time) should crash the process; unhandled rejections (including from an ESM module that does Ideally, a module-level exception from a CJS file should look the same in any version of node, whether or not you're including ESM files. Your last comment implies we might be pretty close to the ideal scenario :-) I'll take a look! |
Since playing around a little with the
I added some tests to check how the process exits, but do not really have any tests checking the error output. EDIT: Only Node >=14 seems to support EDIT: Added some tests checking for unhandled rejections in the stderr. |
@ljharb have you had any time to review the latest changes? |
@lohfu thanks for your patience and for working so thoroughly on this, I really appreciate it! I'll cut a release that includes this soon. |
- [New] Use import() on esm files in supported node versions (#547) - [Refactor] `bin/tape`: separate "preparing of files list" from "require files list" - [Refactor] avoid reassigning arguments - [Refactor] remove unused line, unneeded var initialization; add missing `new` - [Refactor] remove use of legacy `exports` - [Refactor] add names to `Test.prototype` functions - [Deps] update `object-inspect` - [Docs] correct docs for `t.teardown` (#558) - [readme] Another way to create custom reporters (#556) - [readme] remove travis badge; add actions and codecov badges - [Deps] update `glob`, `is-regex`, `object-inspect` - [meta] ensure `not-in-publish`‘s absence does not fail anything - [meta] add `safe-publish-latest`; use `prepublishOnly` script for npm 7+ - [Dev Deps] update `aud`, `eslint` - [Tests] uncaught exceptions and unhandled rejections importing files with bin/tape (#547) - [Tests] exclude examples from coverage - [actions] use `node/install` instead of `node/run`; use `codecov` action
See v5.3.0 |
Just here to say that this is amazing, thank you everyone 🥳 |
This is a proof-of-concept of using
import()
to load ESM files in supported node versions (>= 14, >= 13.2.0, >= 12.17.0). The syntax breakingimport()
call is located in a separate file frombin/tape.js
that is conditionally loaded, preventing older Node versions from crashing.This is currently an unfinished draft. I'm mainly curious to see if you are interested in something like this.
Currently this branch does not use promises to sequentially load the imported files. I have a separate branch, https://github.com/lohfu/tape/blob/esm-support-promises/bin/load-module-esm-support.js#L27-L29, where I have implemented sequential loading but this causes a test repo with a couple of simple test files to fail.
Questions / Caveats
not ok 2 test exited without ending: file 2 test 1
. The tests simply contain a single t.pass and a t.end. All tests in the tape repo pass.${cwd}/package.json
is loaded as I was a little reluctant to add a new dependency to find closest package.json... Is this ok? Do you have any preferences?failed to instrument /home/lohfu/dev/tape/bin/load-module-esm-support.js with error: SyntaxError: 'import' and 'export' may only appear at the top level (14:8)
... Can I update tap to get a newer nyc?import()
. This prevents the use of loaders on other file extensions. In the future an --extension argument will probably be needed in the cli to allow for loaders on custom extensions.